2022 Fiscal Year Final Research Report

A study on the neural mechanisms of speech production and perception using EEG and eye-movement

Research Project

PDF

Project/Area Number	20K11883
Research Category	Grant-in-Aid for Scientific Research (C)
Allocation Type	Multi-year Fund
Section	一般
Review Section	Basic Section 61010:Perceptual information processing-related
Research Institution	Japan Advanced Institute of Science and Technology
Principal Investigator	Dang Jianwu 北陸先端科学技術大学院大学, 先端科学技術研究科, 名誉教授 (80334796)
Co-Investigator(Kenkyū-buntansha)	赤木正人北陸先端科学技術大学院大学, 先端科学技術研究科, 名誉教授 (20242571)
Project Period (FY)	2020-04-01 – 2023-03-31
Keywords	音声生成 / 音声知覚 / 神経学メカニズム / 脳ネットワーク / 音声処理の神経学的モデル
Outline of Final Research Achievements	In this study, we used EEG, eye movement, and speech signals to explore the dynamic characteristics of brain networks during reading and listening to sentences. As a result, we identified top-down processes in several higher cognitive and language areas, including the prefrontal, frontal, and inferior frontal lobes. Higher cognitive and verbal networks were detected with early activation, frequent interactions with the lower visuomotor system, parallel and repetitive interactions in alignment with sentence structure. The brain network activity is clearly different between comprehending continuous speech and listening to nonsensical sentences. It is also possible to analyze brain activity in detail using the non-linearity of deep learning methods. Based on those results, we constructed a neural model of speech production and understanding.
Free Research Field	知覚情報処理
Academic Significance and Societal Importance of the Research Achievements	当面人工知能は知覚知能から認知知能へ発展する段階になっており、音声理解や認知に関する神経学メカニズムの究明はその重要な一環である。本研究はEEGを用いて自然音声の発話と知覚における脳活動を全時間帯域で計測し、連続音声の発話計画、聴覚処理および語彙理解などの複数の脳活動について時間、周波数及び空間上で分析して、これまでにない知見を得た上、従来の二重経路モデルや階層モデルなど音声処理神経モデルに比べ、より精密なモデルを構築した。本研究は、音声理解や認知の側面から人工知能のさらなる発展に大きな貢献があり、言語障害の神経学的な評価やリハビリなどの研究分野へも貢献をもたらすことが期待できる。