An Integrated Neuro-semantic Computational Model to Predict Brain fMRI Responses to Sentence Comprehension

2023 Fiscal Year Final Research Report

• Project/Area Number: 19K12727
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 90030:Cognitive science-related
• Research Institution: Tokyo Institute of Technology
• Principal Investigator: Akama Hiroyuki 東京工業大学, リベラルアーツ研究教育院, 准教授 (60242301)
• Co-Investigator(Kenkyū-buntansha): 粟津 俊二 実践女子大学, 人間社会学部, 教授 (00342684)
• Project Period (FY): 2019-04-01 – 2024-03-31
• Keywords: fMRI / 深層学習 / 機械学習 / 自然言語モデル / 脳機能的連結性

Outline of Final Research Achievements

In machine learning for predicting verbal thoughts from fMRI activation information of the brain, our research has been challenging in terms of 1) overcoming individual differences and inter-individual modeling, and 2) application to more complex sentences and texts. The challenge was to take advantage of the benefits of deep learning. In this study, we were able to construct a method for reconstructing stimulus sentences directly from neural representations of the brain alone. In 1), we also proposed several deep learning algorithms for understanding brain disease cases in order to make the subtleties of cognitive responses in the brain of a specific individual computable, thus making a new contribution to brain cognitive science.

Free Research Field

計算神経認知科学

Academic Significance and Societal Importance of the Research Achievements

本研究が提案する計算神経言語学は、文レベルでの脳内意味処理モデルを構築し、意味の神経表象を生成することで、意味障害に苦しむ患者の言語世界を明らかにしたり、脳外科手術において影響を受ける言語機能も細かく推定できたりするなど、医療など多方面に応用可能である。さらに個人差の克服という視点で開発した深層学習アルゴリズムは、脳のfMRIデータを用い、個別の患者に合った治療法の確立を可能にする。