Study on the hierarchy of cross-modal processing using functional MRI and Deep Learning

2020 Fiscal Year Research-status Report

• Project/Area Number: 19K20390
• Research Institution: National Institute for Physiological Sciences
• Principal Investigator: Pham Quang・Trung 生理学研究所, システム脳科学研究領域, 特任研究員 (60837722)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Keywords: Cognitive Science / Somatosensory / Perception

Outline of Annual Research Achievements

In this study, I seek a computational model which explains how the haptic information is processed into categorical value. I hypothesize that the discrete fragments of haptics information are integrated into visual imagery at the intraparietal sulcus (IPS), then served to the lateral occipital complex (LOC) of the brain. To reveal such hierarchy, I have developed a pneumatic haptic display that can display the fragments of a digit sequentially inside the Magnetic Resonance Imaging (MRI) scanner. I also developed a deep learning model which revealed the unknown hierarchical computation of vision-to-value transformation in the brain.

Current Status of Research Progress

3: Progress in research has been slightly delayed.

Reason

The outbreak of the COVID-19 pandemic limited our resources for conducting the haptic-based FMRI experiments.

Strategy for Future Research Activity

I intend to gather a few local participants for MRI experiments. The participants will be asked to cooperate in safety regulations, with regards to the COVID-19 pandemic.

Causes of Carryover

The influence of the COVID-19 pandemic significantly reduced the amount for attending conference and restricted MRI experiments. In the next fiscal year, I need to use the incurring amount for recruiting the participants (30 participants, estimated) for both visions and haptics experiments and preparing submission for scientific journal.