2023 INTERSPEECH INTERSPEECH 2023

Investigating the cortical tracking of speech and music with sung speech

Abstract

The cortical tracking of speech and music has been primarily investigated separately. Here, we propose a novel paradigm involving sung speech to systematically compare the cortical encoding of sung speech with that of speech and music alone, offering a benchmark for using it in auditory research with ecologically-valid tasks. While this approach will ultimately lead to a variety of neural indices of speech and music processing at various levels of abstraction, the first step is to examine the envelope tracking of sung speech. EEG is recorded from subjects listening to a set of stimuli explicitly designed and built for the comparison: hummed melodies, speech monologues, and sung speech sharing the lyrics with the speech condition and the melody with the music condition. Preliminary analyses using encoding and decoding modeling show robust and consistent acoustic responses across conditions, with the only significant differences exclusively due to melody processing.

🌉 Interdisciplinary Bridge — Healthcare & Medicine and Interdisciplinary and Machine Learning and Speech & Audio
🧭 Keyword Pioneer — cortical tracking
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio