← Back to papers
2009 NIPS NeurIPS 2009

The Wisdom of Crowds in the Recollection of Order Information

Abstract

When individuals independently recollect events or retrieve facts from memory, how can we aggregate these retrieved memories to reconstruct the actual set of events or facts? In this research, we report the performance of individuals in a series of general knowledge tasks, where the goal is to reconstruct from memory the order of historic events, or the order of items along some physical dimension. We introduce two Bayesian models for aggregating order information based on a Thurstonian approach and Mallows model. Both models assume that each individuals reconstruction is based on either a random permutation of the unobserved ground truth, or by a pure guessing strategy. We apply MCMC to make inferences about the underlying truth and the strategies employed by individuals. The models demonstrate a wisdom of crowds" effect, where the aggregated orderings are closer to the true ordering than the orderings of the best individual."

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning
🧭 Keyword Pioneer — order aggregation
🐣 Hot Topic Early Bird — markov chain monte carlo
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning
🌱 Topic Pioneer — Crowdsourcing
📈 Trend Setter — Reasoning

Authors

Mark Steyvers , Brent Miller , Pernille Hemmer , Michael D. Lee

Topics

Artificial Intelligence > Bayesian & Probabilistic > Bayesian Learning Machine Learning > Core Methods > Clustering Machine Learning > Optimization & Theory > Bayesian Inference Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Optimization & Theory > Statistics Artificial Intelligence > Core AI > Reasoning Machine Learning > Bayesian & Probabilistic > Bayesian Inference Machine Learning > Learning Types > Crowdsourcing

Keywords

bayesian inference markov chain monte carlo bayesian model order aggregation mallows model thurstonian model wisdom of crowd order information
Download PDF

Related papers

Solving Stochastic Games 2009
Bilinear classifiers for visual recognition 2009
Zero-shot Learning with Semantic Output Codes 2009
Matrix Completion from Power-Law Distributed Samples 2009
Heavy-Tailed Symmetric Stochastic Neighbor Embedding 2009