• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

An unified framework for audio-visual integration and semantic memory: Neural-computational modelling

Research Project

Project/Area Number 20H04494
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 90030:Cognitive science-related
Research InstitutionTokyo Woman's Christian University

Principal Investigator

Taiji Ueno  東京女子大学, 現代教養学部, 教授 (20748967)

Co-Investigator(Kenkyū-buntansha) 田中 章浩  東京女子大学, 現代教養学部, 教授 (80396530)
石橋 遼  国立研究開発法人情報通信研究機構, 未来ICT研究所脳情報通信融合研究センター, 研究員 (90750266)
Project Period (FY) 2020-04-01 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥17,810,000 (Direct Cost: ¥13,700,000、Indirect Cost: ¥4,110,000)
Fiscal Year 2023: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2022: ¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2021: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2020: ¥8,190,000 (Direct Cost: ¥6,300,000、Indirect Cost: ¥1,890,000)
Keywords視聴覚統合 / 意味記憶 / マガーク効果 / fMRI / 経頭蓋磁気刺激法 / 経頭蓋直流電気刺激法 / 経頭蓋直流電気刺激 / 計算機モデル / 視聴覚相互作用 / TMS / 前部側頭葉 / ニューラルネットワーク
Outline of Research at the Start

本申請計画は、視聴覚統合と意味記憶という、認知科学における2つの大きな研究分野を統合的に説明する単一の横断的なneuro-computationalモデルを提唱することを目的とする。
そのため、fMRI、神経心理学、脳刺激法、コンピューターシミュレーションの4つのアプローチを統合的に用いることにより、方法論的にも領域横断的な統合的理論を提唱していく。

Outline of Final Research Achievements

This study aimed to propose a cognitive neurobiological model that accounts for both audio-visual integration and semantic memory by investigating the common brain regions supporting these functions. A multi-method approach, including fMRI, TMS, and tDCS, revealed the necessity of reassessing the role of the left posterior superior temporal sulcus, which has been considered a consensus in the field of auditory-visual integration. Moreover, the representational searchlight analysis revealed the role of the motor cortex in audio-visual integration. Indeed, past studies have already suggested this region, and interpreted in terms of the motor theory of speech perception. However, due to their use of the conventional fMRI analysis (i.e., subtraction method), there was still a dispute regarding the validity of the motor theory. In contrast, our demonstration by the representational searchlight analysis provided stronger evidence for the motor theory.

Academic Significance and Societal Importance of the Research Achievements

本研究は、これまでコンセンサスとされてきた「視聴覚統合における左脳後部上側頭溝の役割」を支持しない結果を一貫して提示したのみならず、先行研究のデータの再分析に基づいて、これまでの知見が間違っている可能性を示したことに学術的な意義がある。また、差分法fMRI分析の結果を根拠とした「音声知覚の運動理論」についてはこれまで批判があったが、その批判に当てはまらない結果を新しく表象類似性サーチライト分析で示したところに本研究の学術的意義がある。音声知覚の運動理論は、母語・第二言語学習にあたって、聴き取り訓練のために発話訓練をすることが効果的である、などの実践的意義を持つことが期待される。

Report

(5 results)
  • 2023 Annual Research Report   Final Research Report ( PDF )
  • 2022 Annual Research Report
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • Research Products

    (11 results)

All 2024 2023 Other

All Int'l Joint Research (4 results) Journal Article (2 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 2 results,  Open Access: 1 results) Presentation (3 results) (of which Int'l Joint Research: 2 results) Book (2 results)

  • [Int'l Joint Research] University of Leeds(英国)

    • Related Report
      2023 Annual Research Report
  • [Int'l Joint Research] Universitat Autonoma de Barcelona(スペイン)

    • Related Report
      2023 Annual Research Report
  • [Int'l Joint Research] University of Edinburgh/Department of Psychology(英国)

    • Related Report
      2021 Annual Research Report
  • [Int'l Joint Research] University of Edinburgh/Department of Psychology(英国)

    • Related Report
      2020 Annual Research Report
  • [Journal Article] 2020 feels slow, long, and far away: Time distortion due to the COVID‐19 pandemic2024

    • Author(s)
      Castella; Judit, Ueno Taiji, Allen Richard J.
    • Journal Title

      Applied Cognitive Psychology

      Volume: 38 Issue: 2 Pages: 1-13

    • DOI

      10.1002/acp.4182

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Evidence from a within-language comparison in Japanese for orthographic depth theory: Monte Carlo simulations, corpus-based analyses, neural networks, and human experiment2023

    • Author(s)
      Inohara Keisuke、Ueno Taiji
    • Journal Title

      Journal of Memory and Language

      Volume: 132 Pages: 104434-104434

    • DOI

      10.1016/j.jml.2023.104434

    • Related Report
      2023 Annual Research Report 2022 Annual Research Report
    • Peer Reviewed
  • [Presentation] 2020 feels slow, long, and far away: Time distortion due to the pandemic2023

    • Author(s)
      Judit Castella;, Taiji Ueno, Richard Allen
    • Organizer
      Third International Conference of Timing Research Forum
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 乳児の泣き声にみる母語の起源―深層学習モデルを用いた母音弁別学習バイアスの検討―2023

    • Author(s)
      新屋裕太・上野泰治・渡辺はま・多賀厳太郎
    • Organizer
      日本赤ちゃん学会第23回学術集会
    • Related Report
      2023 Annual Research Report
  • [Presentation] Limb Movements and Vocalizations to Music during Crying in Infants at Three- to Four-month-old Age2023

    • Author(s)
      Kenta Ueda, Sotaro Kondoh, Yuta Shinya, Hirotaka Gima, Taiji Ueno, Hama Watanabe, Gentaro Taga, Shinya Fujii
    • Organizer
      The International Congress of Infant Studies 2024
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Book] コロナ禍におけるオンライン教育相談 大家まゆみ・稲垣勉(編)『グローバル時代の教育相談』2024

    • Author(s)
      上野泰治
    • Total Pages
      148
    • Publisher
      ナカニシヤ出版
    • ISBN
      9784779517778
    • Related Report
      2023 Annual Research Report
  • [Book] ことばと記憶 辻幸夫・菅井三実・佐治伸郎 (編) 『シリーズ ことばの認知科学』2024

    • Author(s)
      上野泰治
    • Publisher
      北大路書房
    • Related Report
      2023 Annual Research Report

URL: 

Published: 2020-04-28   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi