Classify Imaginary Mandarin Tones with Cortical EEG Signals

Speech synthesis system based on non-invasive brain-computer interface technology has the potential to restore communication abilities to patients with communication disorders. To this end, electroencephalogram (EEG) based speech imagery technology is fast evolving largely due to its advantages of simple implementation and low dependence on external stimuli. This work studied possible factors accounting for the classification accuracies of EEG-based imaginary Mandarin tones, which has significance to the development of BCI-based Mandarin speech synthesis system. Specially, a Mandarin tone imagery experiment was designed, and this work studied the effects of electrode configuration and tone cuing on accurately classifying four Mandarin tones from cortical EEG signals. Results showed that the involvement of more activated brain regions (i.e., Broca’s area, Wernicke’s area, and primary motor cortex) provided a more accurate classification of imaginary Mandarin tones than that of one specific region. At the tone cue stage, using audio-visual stimuli led to a much stronger and more separable activation of brain regions than using visual-only stimuli. In addition, the classification accuracies of tone 1 and tone 4 were significantly higher than those of tone 2 and tone 3.