Study on linguistic representation and identification of phones from speech imagery EEG.

2022 Fiscal Year Final Research Report

• Project/Area Number: 20K11910
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61020:Human interface and interaction-related
• Research Institution: Toyohashi University of Technology
• Principal Investigator: Nitta Tsuneo 豊橋技術科学大学, 工学(系)研究科(研究院), シニア研究員 (70314101)
• Co-Investigator(Kenkyū-buntansha): 入部 百合絵 愛知県立大学, 情報科学部, 准教授 (40397500) ; 田口 亮 名古屋工業大学, 工学(系)研究科(研究院), 准教授 (70508415) ; 桂田 浩一 東京理科大学, 理工学部情報科学科, 教授 (80324490)
• Project Period (FY): 2020-04-01 – 2023-03-31
• Keywords: 脳波 / 音声想起 / 線形予測分析 / 音節ラベリング / 音素認識

Outline of Final Research Achievements

Speech imagery recognition from electroencephalograms (EEGs) could potentially become a strong contender among non-invasive brain-computer interfaces Is). We extract language representations as the difference of line-spectra of phones by analyzing many EEG signals from the Broca area. Then we extract vowels by using iterative search from hand-labelled short-syllable data. The iterative search process consists of principal component analysis (PCA) that visualizes linguistic representation of vowels through eigen-vectors φ(m), and subspace method (SM) that searches an optimum line-spectrum for redesigning φ(m). The extracted linguistic representation of Japanese vowels /i/ /e/ /a/ /o/ /u/ shows 2 distinguished spectral peaks (P1, P2) in the upper frequency range. The 5 vowels are aligned on the P1-P2 chart. The experiment of 5-vowel recognition is conducted using a data set of 5 subjects and the classifier based on convolutional neural network (CNN) shows 72.6% in average.

Free Research Field

デジタル信号処理，機械学習，音声言語処理

Academic Significance and Societal Importance of the Research Achievements

BCI研究は運動制御等の分野に限られてきたが，今回，音声言語を利用することが初めて可能になった．今回は5母音に限られているが，子音の言語表象が見出されると，脳波によるタイプライターを実現できる．この技術は ，近い将来，ALS患者ほかの方達にとって，生活の質(QoL)を格段に向上するものと期待している．