← Back to papers
2012 NIPS NeurIPS 2012

Learning from the Wisdom of Crowds by Minimax Entropy

Abstract

An important way to make large training sets is to gather noisy labels from crowds of nonexperts. We propose a minimax entropy principle to improve the quality of these labels. Our method assumes that labels are generated by a probability distribution over workers, items, and labels. By maximizing the entropy of this distribution, the method naturally infers item confusability and worker expertise. We infer the ground truth by minimizing the entropy of this distribution, which we show minimizes the Kullback-Leibler (KL) divergence between the probability distribution and the unknown truth. We show that a simple coordinate descent scheme can optimize minimax entropy. Empirically, our results are substantially better than previously published methods for the same problem.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning
📈 Trend Setter — Loss Functions
🧭 Keyword Pioneer — minimax entropy
🐝 Cross-Pollinator — Artificial Intelligence, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning
🐣 Hot Topic Early Bird — probabilistic modeling

Authors

Dengyong Zhou , Sumit Basu , Yi Mao , John C. Platt

Topics

Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Core Methods > Classification Machine Learning > Learning Types > Unsupervised Learning Machine Learning > Learning Types > Weakly Supervised Learning Machine Learning > Optimization & Theory > Loss Functions Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Optimization & Theory > Probabilistic Modeling Machine Learning > Learning Types > Classification

Keywords

probabilistic modeling crowdsourcing label noise kullback-leibler divergence label aggregation crowdsourced learning minimax entropy worker expertise noisy label crowdsourced label
Download PDF

Related papers

Kernel Hyperalignment 2012
Fused sparsity and robust estimation for linear models with unknown variance 2012
Slice sampling normalized kernel-weighted completely random measure mixture models 2012
Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization 2012
Matrix reconstruction with the local max norm 2012