← Back to papers
2020 ICML ICML 2020

Uncertainty Estimation Using a Single Deep Deterministic Neural Network

Abstract

We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. Our approach, deterministic uncertainty quantification (DUQ), builds upon ideas of RBF networks. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models. By enforcing detectability of changes in the input using a gradient penalty, we are able to reliably detect out of distribution data. Our uncertainty quantification scales well to large datasets, and using a single model, we improve upon or match Deep Ensembles in out of distribution detection on notable difficult dataset pairs such as FashionMNIST vs. MNIST, and CIFAR-10 vs. SVHN.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Machine Learning
🧭 Keyword Pioneer — deterministic uncertainty quantification
🐣 Hot Topic Early Bird — out-of-distribution detection
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Deep Learning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning

Authors

Joost van Amersfoort , Lewis Smith , Yee Whye Teh , Yarin Gal

Topics

Artificial Intelligence > Core AI > Interpretability Machine Learning > Core Methods > Classification Machine Learning > Core Methods > Representation Learning Deep Learning Machine Learning > Learning Types > Deep Learning Deep Learning > Learning Types > Deep Learning Machine Learning > Learning Types > Uncertainty Quantification Deep Learning > Models > Deep Learning

Keywords

uncertainty quantification deep learning out-of-distribution detection deep ensemble gradient penalty deterministic uncertainty quantification rbf network out of distribution detection deterministic neural network softmax model
Download PDF

Related papers

Correlation Clustering with Asymmetric Classification Errors 2020
Learning Portable Representations for High-Level Planning 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need 2020
Minimax Pareto Fairness: A Multi Objective Perspective 2020
DeepMatch: Balancing Deep Covariate Representations for Causal Inference Using Adversarial Training 2020