DropFormer: A Dynamic Noise-Dropping Transformer for Speech Emotion Recognition

Speech Emotion Recognition (SER) is an important component for human-computer interaction. Recently, various optimized Transformer variants have been applied to SER. However, most of studies use all the information in the sample and tend to overlook local details, making it difficult to perceive emotional information that is present locally in speech. While there are studies exploring how to utilize local information, their approaches are not flexible enough or are overly complex. To address the issues, we propose DropFormer, a new architecture that focuses only on the emotional segments by dynamically dropping non-emotional information. DropFormer consists of two main components: (1) Drop Attention, proficient in capturing local emotion and highlighting emotion-related segments, (2) Token Dropout Module, capable of dropping tokens lacking emotional information. Experimental results show that our DropFormer achieves state-of-the-art performance on the IEMOCAP and MELD benchmarks.