• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of a language-independent system for automatic evaluation of L2 oral reproduction tasks and the study of its application in foreign language education

Research Project

Project/Area Number 16H03084
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Educational technology
Research InstitutionSoka University (2018-2019, 2021)
Tokyo International University (2016-2017)

Principal Investigator

Yamauchi Yutaka  創価大学, 教育学部, 教授 (30306245)

Co-Investigator(Kenkyū-buntansha) 西川 惠  東海大学, 国際教育センター, 准教授 (10453705)
舟生 日出男  創価大学, 教育学部, 教授 (20344830)
Husky Kay  東京国際大学, 商学部, 准教授 (50237955)
峯松 信明  東京大学, 大学院工学系研究科(工学部), 教授 (90273333)
木下 聖子  創価大学, 糖鎖生命システム融合センター, 教授 (50440235)
染谷 泰正  関西大学, 外国語学部, 教授 (40348454)
川村 明美  東京国際大学, 言語コミュニケーション学部, 教授 (30326996)
Project Period (FY) 2016-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥17,290,000 (Direct Cost: ¥13,300,000、Indirect Cost: ¥3,990,000)
Fiscal Year 2019: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Fiscal Year 2018: ¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2017: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2016: ¥5,590,000 (Direct Cost: ¥4,300,000、Indirect Cost: ¥1,290,000)
Keywords外国語教育 / シャドーイング / 自動評価 / 深層学習 / 音素事後確率系列 / 音響モデル / GOP / DTW / 言語非依存 / 学習者音声 / 外国語 / 多言語 / 英語教育 / 評価 / 総合的熟達度 / 多言語対応 / ティープ・ランニング / 手動評価 / 評価制度 / ディープ・ラーニング / 評価精度 / 言語優勢度
Outline of Final Research Achievements

To improve the oral communication skills of foreign language learners, simultaneous oral reproduction (shadowing) practice is thought to be effective in brushing up learners’ listening comprehension skills and speaking skills and promoting learners’ language processing. A new practice system was developed which can automatically analyze and evaluate simultaneous L2 oral reproduction tasks. A new evaluation algorithm, which applied the posteriorgram to the comparison between the model voice and the learner’s utterance, was employed in this system. The algorithm also made use of a multi-lingual acoustic model consisting of five different languages so that the system could be language-independent. The results of several experiments showed that the new algorithm contributed to the improvement of measurement accuracy and reduction of language dependence in evaluating L2 oral reproduction tasks.

Academic Significance and Societal Importance of the Research Achievements

従来の音声認識を使った音声評価システムでは,モデル音声と学習者音声の直接比較ができず,さらに,言語ごとに音響モデルを用意する必要があった。本研究の自動評価についての新アルゴリズムは,従来の音声認識法と比較して評価精度が向上し、特定言語への非依存性を検討した点が学術的な意義といえる。学習者音声との直接比較が可能となったため,外国語教育分野で活用した場合の学習者へのフィードバックが正確になり,言語ごとに異なる音響モデルを用意する必要がないため多言語を対象とした外国語教育に広く利用できる可能性を含んでいる点が社会的意義といえる。

Report

(5 results)
  • 2021 Final Research Report ( PDF )
  • 2019 Annual Research Report
  • 2018 Annual Research Report
  • 2017 Annual Research Report
  • 2016 Annual Research Report
  • Research Products

    (35 results)

All 2022 2021 2019 2018 2017 2016 Other

All Int'l Joint Research (3 results) Journal Article (14 results) (of which Int'l Joint Research: 3 results,  Peer Reviewed: 8 results) Presentation (18 results) (of which Int'l Joint Research: 12 results,  Invited: 1 results)

  • [Int'l Joint Research] Emory University(米国)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] Melbourne University(オーストラリア)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] Humboldt Universit(ドイツ)

    • Related Report
      2019 Annual Research Report
  • [Journal Article] シャドーイングの自動評価機能を活用した次世代型国際交流サイトの開発と実践2022

    • Author(s)
      山内豊, 木下聖子, 峯松信明
    • Journal Title

      言語教育エキスポ2021発表予稿集

      Volume: 1 Pages: 59-60

    • Related Report
      2019 Annual Research Report
  • [Journal Article] Effects of online language exchange on motivation, cultural awareness, and perception about language learning in a collegiate elementary level Japanese cours2021

    • Author(s)
      Mizuki Mazzotta, Yutaka Yamaguchi
    • Journal Title

      Proceedings of the 36th Annual Conference of the Southeastern Association of Teachers of Japanese

      Volume: 1 Pages: 293-308

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] インタラクション重視の同期型オンライン英語科教育授業の満足度に与える要因分析2021

    • Author(s)
      山内 豊
    • Journal Title

      日本教育工学会研究報告集

      Volume: 3 Pages: 94-97

    • NAID

      130008112874

    • Related Report
      2019 Annual Research Report
  • [Journal Article] 母語話者シャドーイングとそれに基づく「聞き取り易さ」の客観的計測2018

    • Author(s)
      Yutaka Yamauchi, Nobuaki Minematsu, Kayoko Ito, Megumi Nishikawa, KayHusky, Aki Kunikoshi
    • Journal Title

      日本音声学会全国大会発表論文集

      Volume: 1 Pages: 54-59

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] シャドーイング音声自動評価における耐雑音性向上に関する検討2018

    • Author(s)
      椛島優, 齋藤大輔, 峯松信明, 山内豊, 伊藤佳世子
    • Journal Title

      日本音響学会秋季講演論文集

      Volume: 1 Pages: 829-832

    • Related Report
      2018 Annual Research Report
  • [Journal Article] Natives' shadowability as objectively measured comprehensibility of non-native speech2018

    • Author(s)
      Nobuaki Minematsu, Yusuke Inoue, Suguru Kabashima, Daisuke Saito, Yutaka Yamauchi, Kumi Kanamura
    • Journal Title

      Proceedings of the 2nd International Symposium on Applied Phonetics

      Volume: 1 Pages: 28-35

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 母語話者シャドーイングによる「聴き取りやすさ」の客観的計測2018

    • Author(s)
      峯松信明, 山内豊
    • Journal Title

      外国語教育メディア学会全国研究大会発表論文集

      Volume: 1 Pages: 162-163

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 継続的なシャドーイング訓練が総合的熟達力の伸張に及ぼす影響2018

    • Author(s)
      山内豊, 伊藤佳世子, 峯松信明, 坪田康,西川惠
    • Journal Title

      外国語教育メディア学会全国研究大会発表論文集

      Volume: 1 Pages: 160-161

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 音声変形技術を用いたHVTとそれに基づくリスニングの頑健化2018

    • Author(s)
      峯松信明, 山内豊
    • Journal Title

      外国語教育メディア学会全国研究大会発表論文集

      Volume: 1 Pages: 102-103

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Automatic speech quality control of English listening materials and examination of Japanese learners’ listening ability in terms of robustness2018

    • Author(s)
      Haoyu Zhang, Yusuke Inoue, Daisuke Saito, Nobuaki Minematsu, Yutaka Yamauchi and Hinako Masuda
    • Journal Title

      電子情報通信学会音声研究会発表論文集

      Volume: 1 Pages: 1367-1370

    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Journal Article] シャドーイング音声自動評価における耐雑音化と回帰を用いた高精度化2018

    • Author(s)
      椛島優,齋藤大輔,峯松信明,山内豊,伊藤佳世子
    • Journal Title

      情報処理学会研究発表論文集

      Volume: 3 Pages: 1-6

    • Related Report
      2018 Annual Research Report
  • [Journal Article] 母語話者シャドーイングに基づく非母語話者音声の了解性計測に関する予備的検討2018

    • Author(s)
      井上雄介, 椛島優, 齋藤大輔, 峯松信明, 金村久美, 山内豊
    • Journal Title

      日本音響学会春季講演論文集

      Volume: 3 Pages: 1-6

    • Related Report
      2018 Annual Research Report
  • [Journal Article] Investigation of teacher-selected sentences and machine-suggested2017

    • Author(s)
      Yutaka Yamauchi, Junwei Yue, Kayoko Ito and Nobuaki Minematsu
    • Journal Title

      Proceedings of Speech and Language Technology in Education 2017

      Volume: 1 Pages: 30-35

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] 1.語彙サイズ、認識速度、処理の自動化を測定するオンラインL2語彙テストの開発2017

    • Author(s)
      山内豊,峯松信明,川村明美,西川惠,加藤集平
    • Journal Title

      外国語教育メディア学会関東支部 研究紀要

      Volume: 1 Pages: 1-24

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Presentation] コンテンツ・シャドーイングとリピーティングが 英文内容理解に与える影響2021

    • Author(s)
      山内 豊
    • Organizer
      外国語教育メディア学会第60回全国大会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 継続的なシャドーイング練習が第2言語の総合的熟達度向上に及ぼす影響2021

    • Author(s)
      山内 豊
    • Organizer
      世界応用言語学会
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] How much does automatic evaluation based on deep neural network GOP cover prosodic features in L2 oral task assessment?2019

    • Author(s)
      Megumi Nishikawa,Yutaka ,Yamauchi Kayoko Ito, Kay Huskyand Nobuaki Minematsu
    • Organizer
      The American Association for Applied Linguistics Conference (AAAL 2019)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Verification of Cognitive Load Theory: how much does continuous simultaneous oral reproduction training improve L2 overall proficiency?2019

    • Author(s)
      Yutaka Yamauchi, Megumi Nishikawa, Kayoko Ito Kay Husky and Nobuaki Minematsu
    • Organizer
      The American Association for Applied Linguistics Conference (AAAL 2019)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] An experimental study of influence of classroom babble noise on automatic assessment of learners’ shadowing speech2019

    • Author(s)
      Suguru Kabashima, Daisuke Saito, Nobuaki Minematsu, Yutaka Yamauchi, Kayoko Ito
    • Organizer
      The Institute of Electronics, Information and Communication Engineers
    • Related Report
      2018 Annual Research Report
  • [Presentation] Inter-learner shadowing with speech technologies enables automatic and objective measurement of comprehensibility of learners' utterances2019

    • Author(s)
      Megumi Nishikawa,Yutaka ,Yamauchi
    • Organizer
      The American Association for Applied Linguistics Conference (AAAL 2019)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Development of an online L2 vocabulary test and the relationship between spoken and written vocabulary and overall proficiency2018

    • Author(s)
      Yutaka Yamauchi
    • Organizer
      The 5th International Conference on Foreign Language Learning and Teaching
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Measurement accuracy comparison of a new deep learning-based algorithm with a traditional algorithm in automatic L2 oral assessment2017

    • Author(s)
      Yutaka Yamauchi, Kayoko Ito, Kay Husky Nobuaki Minematsu, Megumi Nishikawa, Aki Kunikoshi
    • Organizer
      American Association for Applied Linguistics (AAAL)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Automatic evaluation of simultaneous L2 oral reproduction tasks with a deep learning-based algorithm2017

    • Author(s)
      Yutaka Yamauch, Nobuaki Minematsu, Kayoko Ito, Megumi Nishikawa, KayHusky and Aki Kunikoshi
    • Organizer
      Architectures and Mechanisms of Language Processing
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] The relationship between effective use of working memory and content shadowing performances in L2 processing2017

    • Author(s)
      Yutaka Yamauchi1, Nobuaki Minematsu, Yasumasa Someya, Akemi Kawamura, Megumi
    • Organizer
      The 18th World Congress of Applied Linguistics
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Development and Maintenance of Practical and In-service Systems for Recording Shadowing Utterances and Their Assessment2017

    • Author(s)
      J. Yue, D. Saito, N. Minematsu, Y. Yamauchi & K. Ito
    • Organizer
      peech and Language Technology in Education
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Automatic scoring of shadowing speech based on DNN Posteriors and their DTW2017

    • Author(s)
      J. Yue, F. Shiozawa, S. Toyama, Y. Yamauchi, K. Ito, D. Saito & N. Minematsu
    • Organizer
      INTERSPEECH
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] ディープ・ラーニングに基づく新しいアルゴリズムによるシャドーイング自動評価の精度向上2017

    • Author(s)
      山内 豊・ 峯松 信明・ 伊藤 佳世子・坪田 康・ 西川 惠
    • Organizer
      外国語教育メディア学会
    • Related Report
      2017 Annual Research Report
  • [Presentation] 山内 豊・峯松信明・川村明美・西川惠・加藤集平「意味伝達を重視した英語シャドーイングの新しい自動評価システムの開発と評価」2016

    • Author(s)
      山内 豊・峯松信明・川村明美・西川惠・加藤集平
    • Organizer
      外国語教育メディア学会
    • Related Report
      2016 Annual Research Report
  • [Presentation] Automatic Assessment and Error Detection of Shadowing Speech: Case of English Spoken by Japanese Learners2016

    • Author(s)
      Shuju Shi, Yosuke Kashiwagi, Shohei Toyama, Junwei Yue, Yutaka Yamauchi, Daisuke Saito, Nobuaki Minematsu
    • Organizer
      InterSpeech 2016
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Corpus-based Analysis of Shadowing Speech: Case of L2 English by Japanese Learners2016

    • Author(s)
      Shuju Shi,Yutaka Yamauchi, Nobuaki Minematsu
    • Organizer
      ISAPh2016
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 誤り検出に基づくシャドーイング音声自動評価の精度向上2016

    • Author(s)
      峯松 信明・山内豊
    • Organizer
      外国語教育メディア学会
    • Related Report
      2016 Annual Research Report
  • [Presentation] タブレットPCを使った英文音読の自動評価システムの開発2016

    • Author(s)
      山内 豊・峯松信明・川村明美・西川惠・加藤集平
    • Organizer
      外国語教育メディア学会
    • Related Report
      2016 Annual Research Report

URL: 

Published: 2016-04-21   Modified: 2023-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi