Bridging Emotions Across Languages: Low Rank Adaptation for Multilingual Speech Emotion Recognition

The field of speech emotion recognition (SER) is constantly evolving with the surge in voice data and linguistic diversity. This growth highlights the need for SER systems capable of overcoming language barriers in both linguistic structure and cultural expression of emotions. We envision a SER framework that captures general trends in the expression of emotions, while also modeling language-specific information. Our study investigates low rank adaptation (LoRA) for creating multilingual SER models, applying LoRA in a multilingual context to efficiently adapt pre-trained models to new languages with minimal changes. This enhances cross-lingual adaptability and efficiency of SER systems, refining models to recognize emotions across languages without extensive retraining. In this study, we focus on exploring this method to bridge the gap between English and Taiwanese Mandarin in naturalistic settings, demonstrating strong performance in both languages.