Study of audio-visual integration process revealed by eye-tracking experiment with the "off-screen' sound presentation.

2017 Fiscal Year Final Research Report

• Project/Area Number: 26330311
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Kansei informatics
• Research Institution: Miyagi University
• Principal Investigator: Kayahara Takuro 宮城大学, 事業構想学群(部), 教授 (00345026)
• Project Period (FY): 2014-04-01 – 2018-03-31
• Keywords: シーン認識 / 視聴覚統合 / 視線計測 / サウンドトラック / オフスクリーン音

Outline of Final Research Achievements

In this study, whether the sound induce subjects' gaze and affect meaning of the scene was examined when the subjects was presented with audio-visual stimulus, including the condition in which the sound source is outside of scene ("off-screen" sound condition). The data from eye-track recording revealed that the sound induce subjects' gaze and affect meaning of the scene. Interestingly, in off-screen sound condition, the sound didn't induce subjects' gaze to specific visual object in the scene, but, made meaning of the scene read by subjects more rich and complex.

Free Research Field

知覚・認知心理学