• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Relative assessment of learners' pronunciations based on their bi-directional shadowing and its practical application in class

Research Project

Project/Area Number 20K20407
Project/Area Number (Other) 19H05471 (2019)
Research Category

Grant-in-Aid for Challenging Research (Pioneering)

Allocation TypeMulti-year Fund (2020)
Single-year Grants (2019)
Review Section Studies on the Super-Aging Society
Research InstitutionThe University of Tokyo

Principal Investigator

Minematsu Nobuaki  東京大学, 大学院工学系研究科(工学部), 教授 (90273333)

Co-Investigator(Kenkyū-buntansha) 齋藤 大輔  東京大学, 大学院工学系研究科(工学部), 准教授 (40615150)
Project Period (FY) 2019-06-28 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥25,740,000 (Direct Cost: ¥19,800,000、Indirect Cost: ¥5,940,000)
Fiscal Year 2022: ¥7,020,000 (Direct Cost: ¥5,400,000、Indirect Cost: ¥1,620,000)
Fiscal Year 2021: ¥5,980,000 (Direct Cost: ¥4,600,000、Indirect Cost: ¥1,380,000)
Fiscal Year 2020: ¥6,240,000 (Direct Cost: ¥4,800,000、Indirect Cost: ¥1,440,000)
Fiscal Year 2019: ¥6,500,000 (Direct Cost: ¥5,000,000、Indirect Cost: ¥1,500,000)
Keywords外国語教育 / 音声教育 / 了解性 / 相互シャドーイング / 高齢化・少子化 / 外国人労働者 / ポステリオグラム / シャドーイング / 可解性 / 学習者相互シャドーイング / 高齢化社会支援 / 発音の相対化 / 外国語学習 / 音声情報処理
Outline of Research at the Start

外国人労働者の受け入れが増加する中で,彼らの日本語コミュニケーションを支援する枠組みが求められている。労働者の音声を受け入れ先のスタッフがシャドーすることで,労働者の日本語音声のどこでコミュニケーションが困難となるのかがわかる。また,スタッフが方言話者であれば,スタッフの(母語話者としての)日本語音声を労働者にシャドーさせることで,スタッフの日本語のどこで労働者が困難を抱えているのかを炙り出せる。このように,互いが互いの音声のどこで困難を感じるのかを正直に報告し,それに基づいて相互理解しやすいコミュニケーション戦略を検討するインフラ構築を目指す。

Outline of Final Research Achievements

Language learners' speech is intelligible enough to the learners themselves, but not always intelligible to others. We're proposing a novel method of measuring the intelligibility of L2 speech sequentially and objectively by asking listeners to shadow the L2 speech and quantifying listening breakdowns as shadowing breakdowns found in the shadowing speech. In this study, by using non-native learners as well as native speakers as shadowers (listeners), we investigated the intelligibility of language learners' speech in the following three cases. 1) speaker = Vietnamese learners of Japanese, listener = native speakers of Japanese, 2) speaker = Chinese learners of Japanese, listener = native speakers of Japanese and Chinese learners of Japanese, and 3) speaker = international learners of English, listener = international learners of English.

Academic Significance and Societal Importance of the Research Achievements

本研究では,提示音声を聴取しながら,聴取した通りにその場で復唱するシャドーイングを,学習者音声が即時的にどのように伝わったのかを音声として観測する手法として活用し,学習者と母語話者,更には学習者同士を結びつけ,互いに自分の音声がどう伝わっているのかを分析した。従来,聴取の様子をオンライン計測する手段は脳計測手段に頼っており,極めて高価な機器が必要であったが,本研究では外国語教育的に妥当な方法論で,極めて安価に計測手段を提供できた。学習者にしても,自身の音声が聞かれたその場でどう認識されているのかを初めて知ることができ,実用的価値の高い方法論を提供することができた。

Report

(5 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Research-status Report
  • 2020 Research-status Report
  • 2019 Annual Research Report
  • Research Products

    (50 results)

All 2023 2022 2021 2020 2019

All Journal Article (14 results) (of which Peer Reviewed: 14 results,  Open Access: 14 results) Presentation (32 results) (of which Int'l Joint Research: 8 results,  Invited: 4 results) Patent(Industrial Property Rights) (4 results)

  • [Journal Article] Automatic Prediction of Intelligibility of Words and Phonemes Produced Orally by Japanese Learners of English2023

    • Author(s)
      Zhu Chuanbo、Kunihara Takuya、Saito Daisuke、Minematsu Nobuaki、Nakanishi Noriko
    • Journal Title

      Proc. Spoken Language Technology2022

      Volume: 1 Pages: 1029-1036

    • DOI

      10.1109/slt54892.2023.10023307

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Gradual Improvements Observed in Learners' Perception and Production of L2 Sounds Through Continuing Shadowing Practices on a Daily Basis2022

    • Author(s)
      Kunihara Takuya、Zhu Chuanbo、Minematsu Nobuaki、Nakanishi Noriko
    • Journal Title

      Proc. INTERSPEECH2022

      Volume: 1 Pages: 1303-1307

    • DOI

      10.21437/interspeech.2022-101

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Detection of Learners' Listening Breakdown with Oral Dictation and Its Use to Model Listening Skill Improvement Exclusively Through Shadowing2022

    • Author(s)
      Kunihara Takuya、Zhu Chuanbo、Saito Daisuke、Minematsu Nobuaki、Nakanishi Noriko
    • Journal Title

      Proc. INTERSPEECH2022

      Volume: 1 Pages: 4461-4465

    • DOI

      10.21437/interspeech.2022-440

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Domain Adaptation and Language Conditioning to Improve Phonetic Posteriorgram Based Cross-Lingual Voice Conversion2022

    • Author(s)
      Hsu Pin-Chieh、Minematsu Nobuaki、Saito Daisuke
    • Journal Title

      Proc. APSIPA2022

      Volume: 1 Pages: 950-956

    • DOI

      10.23919/apsipaasc55919.2022.9979918

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Effects of English Shadowing Training Using Unlearned Passages on Listening and Speaking Skills2022

    • Author(s)
      中西 のりこ、峯松 信明、椢原 卓弥
    • Journal Title

      Language Education & Technology

      Volume: 59 Issue: 0 Pages: 77-105

    • DOI

      10.24539/let.59.0_77

    • ISSN
      2185-7792, 2185-7814
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Shadowability of English Learner's Utterances2022

    • Author(s)
      Noriko Nakanishi, Nobuaki Minematsu, Chuanbo Zhu
    • Journal Title

      Journal of Global Communication Studies

      Volume: 7 Pages: 29-44

    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Lexical Density Analysis of Word Productions in Japanese English Using Acoustic Word Embeddings2021

    • Author(s)
      Shintaro Ando, Nobuaki Minematsu, Daisuke Saito
    • Journal Title

      Proc. INTERSPEECH2021

      Volume: 1 Pages: 4433-4437

    • DOI

      10.21437/interspeech.2021-853

    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Multi-Granularity Annotation of Instantaneous Intelligibility of Learners' Utterances Based on Shadowing Techniques2021

    • Author(s)
      Chuanbo Zhu; Ryo Hakoda; Daisuke Saito; Nobuaki Minematsu; Noriko Nakanishi; Tazuko Nishimura
    • Journal Title

      Proc. ASRU2021

      Volume: - Pages: 1071-1078

    • DOI

      10.1109/asru51503.2021.9688270

    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Tensor Factor Analysis for Arbitrary Speaker Conversion2020

    • Author(s)
      Daisuke Saito, Nobuaki Minematsu, Keikichi Hirose
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E103.D Issue: 6 Pages: 1395-1405

    • DOI

      10.1587/transinf.2019EDP7166

    • NAID

      130007850060

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2020-06-01
    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Shadowability Annotation with Fine Granularity on L2 Utterances and Its Improvement with Native Listeners’ Script-shadowing2020

    • Author(s)
      Zhenchao LIN, Ryo TAKASHIMA, Daisuke SAITO, Nobuaki MINEMATSU, Noriko NAKANISHI
    • Journal Title

      Proc. INTERSPEECH

      Volume: 1 Pages: 3865-3869

    • DOI

      10.21437/interspeech.2020-2550

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Optimized Prediction of Fluency of L2 English Based on Interpretable Network Using Quantity of Phonation and Quality of Pronunciation2020

    • Author(s)
      Yang Shen, Ayano Yasukagawa, Daisuke Saito, Nobuaki Minematsu, Kazuya Saito
    • Journal Title

      Proc. IEEE Workshop on Spoken Language Technology

      Volume: 1 Pages: 698-704

    • DOI

      10.1109/slt48900.2021.9383458

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Analysis of Native Listeners’ Facial Microexpressions While Shadowing Non-Native Speech -- Potential of Shadowers’ Facial Expressions for Comprehensibility Prediction --2019

    • Author(s)
      Tasavat Trisitichoke, Shintaro Ando, Daisuke Saito, Nobuaki Minematsu
    • Journal Title

      Proceedings of INTERSPEECH

      Volume: - Pages: 1861-1865

    • DOI

      10.21437/interspeech.2019-1953

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Native Listeners' Shadowing of Non-native Utterances as Spoken Annotation Representing Comprehensibility of the Utterances2019

    • Author(s)
      Zhenchao Lin, Yusuke Inoue, Tasavat Trisitichoke, Shintaro Ando, Daisuke Saito, Nobuaki Minematsu
    • Journal Title

      Proceedings of SLaTE

      Volume: - Pages: 43-47

    • DOI

      10.21437/slate.2019-8

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Large Collection of Sentences Read Aloud by Vietnamese Learners of Japanese and Native Speaker’s Reverse Shadowings2019

    • Author(s)
      Shintaro Ando, Zhenchao Lin, Tasavat Trisitichoke, Yusuke Inoue, Fuki Yoshizawa, Daisuke Saito, Nobuaki Minematsu
    • Journal Title

      Proceedings of O-COCOSDA

      Volume: - Pages: 1-6

    • DOI

      10.1109/o-cocosda46868.2019.9041215

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] Effects of karaoke shadowing on ELF learners’ segmental and prosodic features2023

    • Author(s)
      Noriko NAKANISHI, Nobuaki MINEMATSU
    • Organizer
      CamTESOL2023
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Automatic prediction of word- and phoneme-unit intelligibility of Japanese learners’ L2 English2023

    • Author(s)
      Nobuaki MINEMATSU, Noriko NAKANISH
    • Organizer
      AAAL2023
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 英語絵本読み聞かせ訓練をタスクとした学習者音声の韻律分析とその改善2023

    • Author(s)
      正田千宙, 高英翔, 峯松信明, 齋藤大輔, 中西のりこ
    • Organizer
      音響学会春季研究発表会
    • Related Report
      2022 Annual Research Report
  • [Presentation] 音節を単位とした発音空間のエントロピーに着眼した母語・非母語発音分析2023

    • Author(s)
      勝瑞雄介,峯松信明,齋藤大輔,中西のりこ
    • Organizer
      音響学会春季研究発表会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Correlation analysis between L2 learners’while-listening facial expressions and their listening dysfluency2023

    • Author(s)
      Haokun Yang, Nobuaki Minematsu, Daisuke Saito
    • Organizer
      音響学会春季研究発表会
    • Related Report
      2022 Annual Research Report
  • [Presentation] シャドーイングを用いた英語学習者の聞き淀み計測とその自動推定2023

    • Author(s)
      高英翔,峯松信明,齋藤大輔,中西のりこ
    • Organizer
      音響学会春季研究発表会
    • Related Report
      2022 Annual Research Report
  • [Presentation] Objective comparison of the effects of L2 shadowing on learners’ speech perception and production2022

    • Author(s)
      Nobuaki MINEMATSU, Noriko NAKANISHI
    • Organizer
      New Sounds 2022
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 外国語学習における効率的な韻律学習とその技術支援2022

    • Author(s)
      峯松信明
    • Organizer
      社会言語科学会大会シンポジウム
    • Related Report
      2022 Annual Research Report
    • Invited
  • [Presentation] 音声分析・認識・合成・評価技術が組み込まれた外国語音声教材の開発を支援するオーサリングシステムの開発2022

    • Author(s)
      峯松信明,中西のりこ,井上雄介
    • Organizer
      日本音声学会研究例会シンポジウム
    • Related Report
      2022 Annual Research Report
  • [Presentation] 学習者音声に対する音素・単語を単位とした明瞭度自動推定 シャドーイングを用いた評価者の聞き淀みの計測とモデル化2022

    • Author(s)
      峯松信明,中西のりこ
    • Organizer
      外国語教育メディア学会全国研究大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] 教師なし音声分割と単語埋め込み空間を用いたL2音声の競合単語分析2022

    • Author(s)
      勝瑞雄介,峯松信明,齋藤大輔,中西のりこ
    • Organizer
      音響学会秋季研究発表会
    • Related Report
      2022 Annual Research Report
  • [Presentation] 英語学習者音声の韻律を対象とした自動可視化・採点とその効果2022

    • Author(s)
      峯松信明,中西のりこ,高英翔,崔ジェヒョン,勝瑞雄介
    • Organizer
      日本音声学会全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] L2 音声に対する瞬時的了解度計測を目的とした シャドーイング音声コーパスの構築2022

    • Author(s)
      峯松信明,朱伝博,ダンチャン・ジャバオ,齋藤大輔,中西のりこ
    • Organizer
      音響学会音声研究会
    • Related Report
      2021 Research-status Report
  • [Presentation] Sequential prediction of instantaneous intelligibility in Japanese-accented English utterances2022

    • Author(s)
      Chuanbo Zhu, Takuya Kunihara, Yusuke Shozui, Daisuke Saito, Nobuaki Minematsu, and Noriko Nakanishi
    • Organizer
      音響学会音声研究会
    • Related Report
      2021 Research-status Report
  • [Presentation] 学習者音声に対する母語話者シャドーイングを用いた聴取崩れに関する要因分析2022

    • Author(s)
      ダンチャン・ジャバオ,朱伝博,齋藤大輔,峯松信明,金村久美
    • Organizer
      日本音響学会春季研究発表会
    • Related Report
      2021 Research-status Report
  • [Presentation] 話す→聞く→伝わるように話す→常時聞いて話すための外国語学習環境構築2022

    • Author(s)
      峯松信明
    • Organizer
      日本音響学会春季研究発表会
    • Related Report
      2021 Research-status Report
    • Invited
  • [Presentation] モデル音声への接近と脱落及びモデル音声の知覚に着目した学習者シャドーイング音声の分析2021

    • Author(s)
      椢原卓弥,朱伝博,齋藤大輔,峯松信明,中西のりこ
    • Organizer
      日本音響学会秋季研究発表会
    • Related Report
      2021 Research-status Report
  • [Presentation] Objective and Semi-automatic Measurement of Smoothness of Instantaneous Understanding of L2 English Speech2021

    • Author(s)
      Chuanbo Zhu, Nobuaki Minematsu, Noriko Nakanishi
    • Organizer
      PSLLT2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Objective measurement of instantaneous intelligibility of L2 utterances based on shadowing2021

    • Author(s)
      Nobuaki Minematsu, Ryo Hakoda, Chuanbo Zhu, Noriko Nakanishi, Tazuko Nishimura, Daisuke Saito
    • Organizer
      ISAPh2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] 英語学習者と日本語学習者音声に対する多角的評価に関する実験的検討2021

    • Author(s)
      椢原卓弥,藩陽,斎藤大輔,峯松信明,斉藤一弥,ボイクマン総子
    • Organizer
      音響学会春季全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] Multi-granularity analysis of online intelligibility of L2 speech based on reverse shadowing2021

    • Author(s)
      Chuanbo Zhu, Ryo Hakoda, Daisuke Saito, Nobuaki Minematsu, Noriko Nakanishi
    • Organizer
      音響学会春季全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] 学習者間相互シャドーイングの実現に向けた音声分析条件と 発音教示生成に関する実験的検討2020

    • Author(s)
      青谷 和真,安藤 慎太郎,井上 雄介,齋藤 大輔,峯松 信明
    • Organizer
      情報処理学会音声言語処理研究会
    • Related Report
      2020 Research-status Report
  • [Presentation] 逆シャドーイングに基づく可解性の推定とその応用2020

    • Author(s)
      峯松 信明,青谷 和真,林 振超
    • Organizer
      日本語音声コミュニケーション学会
    • Related Report
      2020 Research-status Report
  • [Presentation] Shadowability-based Analysis of Behaviors of Listeners with Diverse Language Backgrounds when Listening to Japanese English Utterances2020

    • Author(s)
      Chuanbo ZHU, Ryo TAKASHIMA, Shintaro ANDO, Zhenchao LIN, Daisuke SAITO, Nobuaki MINEMATSU, Noriko NAKANISHI
    • Organizer
      音響学会秋季全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] Analyses on Instantaneous Perception of Japanese English by Listeners with Various Language Profiles2020

    • Author(s)
      Chuanbo Zhu, Zhenchao Lin, Nobuaki Minematsu, Noriko Nakanishi
    • Organizer
      音声学会全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] Acoustic Word Embedding を用いた L2 単語音声の語彙密度分析2020

    • Author(s)
      安藤慎太郎,齋藤大輔,峯松信明
    • Organizer
      音響学会秋季全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] 日本人による英語自然発声を対象とした流暢さの高精度自動スコアリングの実現2020

    • Author(s)
      安ヵ川彩乃,安藤慎太郎,紺野瑛介,林振超,井上雄介,齋藤大輔,峯松信明,斉藤一弥
    • Organizer
      電子情報通信学会音声研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 韻律的特徴・ラベルを用いたDNN音響モデルに基づく英語発話時の韻律制御に対する自動推定2020

    • Author(s)
      瀋陽,安藤慎太郎,峯松信明,齋藤大輔,小橋川哲
    • Organizer
      電子情報通信学会音声研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 日本語end-to-end音声合成を用いた韻律シンボル教示とその音響的実現に関する音声教育的考察2020

    • Author(s)
      吉澤風希,熊野正,峯松信明,栗原清
    • Organizer
      電子情報通信学会音声研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] INFLUENCE OF CONTENT VARIATIONS ON SMOOTHNESS OF NATIVE SPEAKERS’ REVERSE SHADOWING2019

    • Author(s)
      Tasavat Trisitichoke, Shintaro Ando, Yusuke Inoue, Daisuke Saito, Nobuaki Minematsu
    • Organizer
      International Congress on Phonetic Scienceson
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Natives' shadowability as objectively measured comprehensibility of non-native speech2019

    • Author(s)
      Nobuaki Minematsu
    • Organizer
      CASS JSPS joint symposium on linguistics
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] How can speech technologies support learners to improve their skills of speaking, listening, conversation, and more?2019

    • Author(s)
      Nobuaki Minematsu
    • Organizer
      ROCLING2019
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Patent(Industrial Property Rights)] 外国語音声の知覚と聴音に関する多角的評価のための循環型シャドーイング教材2022

    • Inventor(s)
      峯松信明,椢原卓弥,朱伝博,箱田 峻,中西のりこ,西村多寿子
    • Industrial Property Rights Holder
      峯松信明,椢原卓弥,朱伝博,箱田 峻,中西のりこ,西村多寿子
    • Industrial Property Rights Type
      特許
    • Filing Date
      2022
    • Related Report
      2022 Annual Research Report
  • [Patent(Industrial Property Rights)] 音声に対する瞬時的了解度の系列推定技術2022

    • Inventor(s)
      峯松信明,朱伝博,中西のりこ
    • Industrial Property Rights Holder
      峯松信明,朱伝博,中西のりこ
    • Industrial Property Rights Type
      特許
    • Filing Date
      2022
    • Related Report
      2022 Annual Research Report
  • [Patent(Industrial Property Rights)] 音声に対する瞬時的了解度の系列推定技術2021

    • Inventor(s)
      峯松信明
    • Industrial Property Rights Holder
      峯松信明
    • Industrial Property Rights Type
      特許
    • Filing Date
      2021
    • Related Report
      2021 Research-status Report
  • [Patent(Industrial Property Rights)] 外国語音声の知覚と調音に関する多角的評価のための循環型シャドーイング教材2021

    • Inventor(s)
      峯松信明
    • Industrial Property Rights Holder
      峯松信明
    • Industrial Property Rights Type
      特許
    • Filing Date
      2021
    • Related Report
      2021 Research-status Report

URL: 

Published: 2019-07-04   Modified: 2024-03-26  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi