Representation of visual and non-visual object attributes in the brain and artificial neural networks

2022 Fiscal Year Final Research Report

• Project/Area Number: 17H01756
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Cognitive science
• Research Institution: National Institute for Physiological Sciences
• Principal Investigator: Goda Naokazu 生理学研究所, システム脳科学研究領域, 助教 (30373195)
• Co-Investigator(Kenkyū-buntansha): 蒲池 みゆき 工学院大学, 情報学部(情報工学部), 教授 (70395101)
• Project Period (FY): 2017-04-01 – 2022-03-31
• Keywords: 視覚 / 多感覚 / 質感 / 深層学習

Outline of Final Research Achievements

Our brain effortlessly recognizes not only visual but also non-visual attributes of objects and environments just by looking at them. To delve into this intriguing neural processing, we analyzed the relationship between the psychological/neural representations of non-visual attributes of objects and the image features derived from deep convolutional neural networks (DNN). We discovered DNN features valuable for recognizing non-visual attributes and developed prototype image synthesis techniques that manipulate non-visual impressions using those features. Additionally, we created a virtual reality environment capable of manipulating visual and non-visual features, revealing new visual-somatosensory interactions within the environment. These techniques contribute to advancing our understanding of the brain's multisensory nature.

Free Research Field

神経科学

Academic Significance and Societal Importance of the Research Achievements

本研究は、未だ解明されていない脳の情報処理を理解するために、現在成功を収めている深層畳込みニューラルネットワーク(DNN)を利用する試みである。脳神経科学研究だけでなくDNNの研究開発においても、複数の感覚（視覚や触覚など）情報を統合的に扱うアプローチはまだこれからのテーマであり、本研究で得られた知見や技術は、これら両分野の研究の進展に貢献することが期待される。