← Back to papers
2013 NIPS NeurIPS 2013

Compete to Compute

Abstract

Local competition among neighboring neurons is common in biological neural networks (NNs). We apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

🌉 Interdisciplinary Bridge — Deep Learning and Machine Learning
📈 Trend Setter — Continual Learning
🧭 Keyword Pioneer — catastrophic forgetting
🐣 Hot Topic Early Bird — catastrophic forgetting
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Deep Learning, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio

Authors

Rupesh K Srivastava , Jonathan Masci , Sohrob Kazerounian , Faustino Gomez , Jürgen Schmidhuber

Topics

Machine Learning > Core Methods > Classification Machine Learning > Learning Types > Continual Learning Deep Learning > Architectures > Neural Networks Artificial Intelligence > Core AI > Adversarial Learning Deep Learning > Learning Types > Deep Learning

Keywords

continual learning catastrophic forgetting feature learning deep learning local competition multilayer neural networks backprop training gradient-based learning neural network
Download PDF

Related papers

Latent Structured Active Learning 2013
On Flat versus Hierarchical Classification in Large-Scale Taxonomies 2013
Generalized Method-of-Moments for Rank Aggregation 2013
Third-Order Edge Statistics: Contour Continuation, Curvature, and Cortical Connections 2013
Accelerated Mini-Batch Stochastic Dual Coordinate Ascent 2013