Vaccinating SER to Neutralize Adversarial Attacks with Self-Supervised Augmentation Strategy

Speech emotion recognition (SER) is being actively developed in multiple real-world application scenarios, and users tend to become intimately connected to these services. However, most existing SER models are vulnerable against a growing diverse set of adversarial attacks. The degraded performances can lead to dreadful user experiences. In this work, we propose a self-supervised augmentation defense (SSAD) strategy to learn a single purify network acts as a general front-end to neutralize adversarial distortions without knowing the types of attack beforehand. We show that our approach can robustly defend against two different gradient-based attacks at various intensities on the well-known IEMOCAP. Further, by examining metrics of protection efficacy and recovery rate, our approach shows a consistent protection behavior to prevent adverse outcomes and is capable to recover samples that are wrongly-predicted before purification.