Decoding Covert Speech from EEG Using Multi-receptive Field Convolutional Neural Networks

Abstract

Since speech is the most natural form of human communication, it can be used in brain-computer interface (BCI) systems to decode covert speech from electroencephalography (EEG). It is known that several frequency bands of EEG are applied to the mechanism of imagining speaking a voice, as in the mechanism of speaking a voice. This study proposes a method to decode imagined speech from EEG through a signal decomposition method and a multi-receptive field convolutional neural network (MRF-CNN). In the first part of this dissertation, the decoding of covert speech was validated using signal decomposition methods, which extract specific frequency bands of EEG and one-dimensional MRF-CNN. The noise-assisted multivariate empirical mode decomposition method performed better than the wavelet packet decomposition in decomposing EEG into particular frequency bands. EEG in the gamma band showed better decoding performance of imagined speech than other frequency bands. Still, as known, the other frequency bands also showed above-average imagined speech decoding performance. In other words, if signal decomposition can be elaborated, it can help decode covert speech from EEG. The second part validated three-dimensional spatio-temporal MRF-CNN to decode covert speech from EEG. The raw EEG signal was represented in three dimensions through three-dimensional mapping, and the imagined speech was decoded through a spatio-temporal MRF-CNN consisting of three temporal MRF-CNN. The three-dimensional spatio-temporal MRF-CNN showed excellent classification accuracy compared to other raw signal methods. The findings suggest that signal decomposition methods and MRF-CNN can decode imagined speech from EEG.