Pronunciation training by applying the visualization of pronunciation-map and articulatory gesture based on articulatory-feature extraction

2015 Fiscal Year Final Research Report

• Project/Area Number: 25280128
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Partial Multi-year Fund
• Section: 一般
• Research Field: Learning support system
• Research Institution: Waseda University
• Principal Investigator: Nitta Tsuneo 早稲田大学, グリーン・コンピューティング・システム研究機構, 教授 (70314101)
• Co-Investigator(Kenkyū-buntansha): Iribe Yurie 愛知県立大学, 情報科学部, 講師 (40397500) ; Hayashi Ryoko 神戸大学, 大学院国際文化学研究科, 教授 (20347785) ; Kawai Goh 北海道大学, 外国語教育センター, 教授 (70314101) ; Katsurada Kouichi 東京理科大学, 理工学部, 准教授 (80324490)
• Project Period (FY): 2013-04-01 – 2016-03-31
• Keywords: 発音学習 / 調音特徴抽出 / 発音マップ / 調音ジェスチャ / HTML5

Outline of Final Research Achievements

A novel computer-assisted pronunciation training (CAPT) system has been developed by extracting articulatory features (AF) from speech to correct pronunciation. Research items focused are (1) improvement of AF extractor performance, (2) a subsystem to plot AF on a pronunciation map, (3) a subsystem to visualize learner’s articulatory gesture by comparing it with the correctly pronounced gesture, (4) a web-based CAPT system. Multiple linear mapping is applied for speech-to- phoneme recognition. In the web-based subsystems and the CAPT system, HTML5 that can handle audio and video data is introduced.

Free Research Field

ヒューマンインタフェース，マルチメディア信号処理，音声言語処理，マルチモーダル対話