2022 Fiscal Year Final Research Report
Effects of simultaneous processing of visual and auditory stimuli in second language: Evidence from reaction time, eye-tracking, and brain activity
| Project/Area Number |
20K13103
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 02100:Foreign language education-related
|
|---|
| Research Institution | Nagoya City University (2022)
Aichi Shukutoku University (2020-2021) |
|---|
Principal Investigator |
Kajiura Mayumi 名古屋市立大学, 大学院人間文化研究科, 准教授 (70849025)
|
|---|
| Project Period (FY) |
2020-04-01 – 2023-03-31
|
| Keywords | 第二言語習得 / マルチモーダルインプット / 第二言語リスニング / 視線計測 / NIRS / 脳活動計測 / オンライン短期集中リスニングトレーニング |
|---|
| Outline of Final Research Achievements |
In four different online 6-day intensive listening training experiments using reading comprehension texts and audio, 143 people participated, and it was found that training with multimodal input, such as simultaneous presentation of audio and reading comprehension texts, was significantly more effective for learning than training with repeated listening only. The results of the eye movement experiment in which 32 native speakers and 53 non-native speakers participated when reading and speech were presented simultaneously revealed that whether multimodal stimuli promote comprehension depends on the reader factor and presentation method. In addition to that, the brain experiment in which 14 non-native speakers participated showed that the brain activity also varies depending on learner characteristics. The multi-modal learning method such as using video captions has different effects depending on both learner (e.g., proficiency) and input (e.g., speech speed) characteristics.
|
| Free Research Field |
外国語教育
|
| Academic Significance and Societal Importance of the Research Achievements |
ビデオキャプション等外国語のマルチメディア学習方法は効果が高いとされているが、マルチモーダルな情報入力処理を行動データのみならず、眼球運動、脳活動からも分析した研究は少ない。これら３側面から検証した結果、学習者の読解速度が提示音声よりも速いか遅いかにより、マルチモーダル入力効果が阻害される可能性があり、音声提示速度や学習者要因がインプットの効果に影響を与えることを示した点で、外国語学習研究分野に貢献できたと考える。同時提示される音声は読解を促進、または阻害する場合もあることから、脳内には視覚処理、聴覚処理と異なるチャネルを持つ可能性があり、ワーキングメモリ仕組み解明への一助となったと考える。
|
