Study on the hierarchy of cross-modal processing using functional MRI and Deep Learning

2019 Fiscal Year Research-status Report

• Project/Area Number: 19K20390
• Research Institution: National Institute for Physiological Sciences
• Principal Investigator: Pham Quang・Trung 生理学研究所, システム脳科学研究領域, 特任研究員 (60837722)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Keywords: Cognitive science / Somatosensory / Haptics / Perception

Outline of Annual Research Achievements

In this study, I seeks for a comprehensive computational model by which the haptic information is processed into categorical value. It has been known that the Iateral occipital complex (LOC) of the brain is engaged in both vision and haptic processing. However, different from the vision, the haptics information is usually a group of fragments provided sequentially. Hence, they must be integrated at some areas. I hypothesize that the intraparietal sulcus (IPS) plays as the middle region between primary somatosensory area (S1) and the LOC, reconstructs the visual imagery from the fragments of haptic information. By that way, the ventral stream pathway which contributes to the human's object classification, can lately process it in the same manner as visual information. To observe such activity of IPS, I have developed a specific pneumatic haptic display that can be used to display the fragments of a digit sequentially inside the Magnetic Resonance Imaging (MRI) scanner. A similar experiment has been prepared for examining the ability of visions in reconstructing visual imagery from visual fragments. Finally, I am developing a deep learning model to demonstrate how the computation occurs hierarchically in the brain during these tasks.

Current Status of Research Progress

3: Progress in research has been slightly delayed.

Reason

The experiment design and device preparation were slightly longer than expected. Moreover, it has not been possible to recruit any participants for the MRI experiment due to the influence of the new coronavirus.

Strategy for Future Research Activity

I intend to conduct the MRI experiments with a controlled closed group of colleagues for preliminarily result. Then I will verified the result with a extended group of participants in the future if possible.

Causes of Carryover

At the mean time, the required amount for preparing MRI experiments was less than expected. However, in the next fiscal year, I need to use the incurring amount for hiring an technical supporter and recruiting about twenty participants for both visions and haptics experiments.