CADE: Continual Weakly-supervised Video Anomaly Detection with Ensembles

Video anomaly detection (VAD) has long been studied as a crucial problem in public security and crime prevention. In recent years, weakly-supervised VAD (WVAD) has attracted considerable attention due to its easy annotation process and promising research results. While existing WVAD methods mainly tackle static datasets, the possibility that the domain of data can vary has been neglected. To adapt such domain shift, the continual learning (CL) perspective is required because otherwise additional training using only newly incoming data could easily cause performance degradation for previous data, i.e., forgetting. Therefore, we propose a brand-new approach, called Continual Anomaly Detection with Ensembles (CADE) that is the first work combining CL and WVAD viewpoints. Specifically, CADE uses the Dual-Generator (DG) to address data imbalance and label uncertainty in WVAD. We also found that forgetting exacerbates the "incompleteness" where the model becomes biased towards certain anomaly modes, leading to missed detections of various anomalies. To address this, we propose a Multi-Discriminator (MD) ensemble that captures missed anomalies in past scenes due to forgetting, using multiple models. Extensive experiments show that CADE significantly outperforms existing VAD methods on the common multi-scene VAD datasets, such as the ShanghaiTech, UCF-Crime, and Charlotte Anomaly Dataset.