← Back to papers
2006 NIPS NeurIPS 2006

In-Network PCA and Anomaly Detection

Abstract

We consider the problem of network anomaly detection in large distributed systems. In this setting, Principal Component Analysis (PCA) has been proposed as a method for discover- ing anomalies by continuously tracking the projection of the data onto a residual subspace. This method was shown to work well empirically in highly aggregated networks, that is, those with a limited number of large nodes and at coarse time scales. This approach, how- ever, has scalability limitations. To overcome these limitations, we develop a PCA-based anomaly detector in which adaptive local data (cid:2)lters send to a coordinator just enough data to enable accurate global detection. Our method is based on a stochastic matrix perturba- tion analysis that characterizes the tradeoff between the accuracy of anomaly detection and the amount of data communicated over the network.

🚀 Conference Pioneer — NIPS 2006
🌱 Topic Pioneer — Information Retrieval
🌉 Interdisciplinary Bridge — Computer Science and Computer Vision and Data Science & Analytics and Machine Learning
📈 Trend Setter — Anomaly Detection
🧭 Keyword Pioneer — principal component analysis
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Speech & Audio
🐣 Hot Topic Early Bird — dimensionality reduction

Authors

Ling Huang , Xuanlong Nguyen , Minos Garofalakis , Michael I. Jordan , Anthony Joseph , Nina Taft

Topics

Machine Learning > Core Methods > Classification Machine Learning > Core Methods > Representation Learning Computer Vision > Analysis > Anomaly Detection Data Science & Analytics > Methods > Data Mining Data Science & Analytics > Applications > Disease Surveillance Computer Science > Applications > Cybersecurity Computer Science > Applications > Information Retrieval Machine Learning > Core Methods > Dimensionality Reduction Data Science & Analytics > Applications > Risk Management Machine Learning > Core Methods > Anomaly Detection Machine Learning > Learning Types > Out-of-Distribution Detection

Keywords

dimensionality reduction anomaly detection principal component analysis network anomaly detection statistical analysis network monitoring subspace tracking matrix perturbation distributed system stochastic perturbation stochastic matrix perturbation
Download PDF

Related papers

Temporal Coding using the Response Properties of Spiking Neurons 2006
Parameter Expanded Variational Bayesian Methods 2006
Effects of Stress and Genotype on Meta-parameter Dynamics in Reinforcement Learning 2006
Ordinal Regression by Extended Binary Classification 2006
Blind source separation for over-determined delayed mixtures 2006