← Back to papers
2006 NIPS NeurIPS 2006

Information Bottleneck Optimization and Independent Component Extraction with Spiking Neurons

Abstract

The extraction of statistically independent components from high-dimensional multi-sensory input streams is assumed to be an essential component of sensory processing in the brain. Such independent component analysis (or blind source separation) could provide a less redundant representation of information about the external world. Another powerful processing strategy is to extract preferentially those components from high-dimensional input streams that are related to other information sources, such as internal predictions or proprioceptive feedback. This strategy allows the optimization of internal representation according to the infor- mation bottleneck method. However, concrete learning rules that implement these general unsupervised learning principles for spiking neurons are still missing. We show how both information bottleneck optimization and the extraction of inde- pendent components can in principle be implemented with stochastically spiking neurons with refractoriness. The new learning rule that achieves this is derived from abstract information optimization principles.

🚀 Conference Pioneer — NIPS 2006
🌉 Interdisciplinary Bridge — Interdisciplinary and Machine Learning
📈 Trend Setter — Self-Supervised Learning
🧭 Keyword Pioneer — sensory processing
🐣 Hot Topic Early Bird — unsupervised learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Speech & Audio

Authors

Stefan Klampfl , Wolfgang Maass , Robert A. Legenstein

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Learning Types > Self-Supervised Learning Machine Learning > Learning Types > Unsupervised Learning Interdisciplinary > Cognitive Science > Perception Machine Learning > Core Methods > Dimensionality Reduction Machine Learning > Optimization & Theory > Information Theory Machine Learning > Core Methods > Feature Learning Deep Learning > Learning Types > Self-Supervised Learning

Keywords

unsupervised learning representation learning blind source separation independent component analysis information bottleneck sensory processing spiking neuron spiking neural network
Download PDF

Related papers

Temporal Coding using the Response Properties of Spiking Neurons 2006
Parameter Expanded Variational Bayesian Methods 2006
Effects of Stress and Genotype on Meta-parameter Dynamics in Reinforcement Learning 2006
Ordinal Regression by Extended Binary Classification 2006
Blind source separation for over-determined delayed mixtures 2006