• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Real-time Visualization of Pronunciation on an IPA Chart based on Articulatory Feature Extraction

Research Project

Project/Area Number 15K00487
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Learning support system
Research InstitutionAichi Prefectural University

Principal Investigator

Iribe Yurie  愛知県立大学, 情報科学部, 准教授 (40397500)

Co-Investigator(Kenkyū-buntansha) 新田 恒雄  早稲田大学, グリーン・コンピューティング・システム研究機構, その他(招聘研究員) (70314101)
Project Period (FY) 2015-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
Fiscal Year 2017: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2016: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2015: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Keywords発音訓練 / 調音運動 / IPAチャート / 発話訓練 / 調音特徴 / リンキング
Outline of Final Research Achievements

We have been developing a pronunciation training system to evaluate and correct learner's pronunciation by extracting articulatory-features (AFs). In this paper, we propose a novel pronunciation training system that can plot the place and manner of articulation of learner's pronunciation on an International Phonetic Alphabet (IPA) chart in real time. First, the proposed system converts input speech into AF-sequences by using multi-layer neural networks. Then, the AF-sequences are converted into x-y coordinates and plotted on an IPA chart to show his/her articulation in real time. Lastly, we investigate plotting accuracy on the IPA chart through experimental evaluation. The correct rates of AFs to every English phoneme averages 93%, and that of unique Japanese phonemes averages about 96%. The results clearly show that the proposed system can extract AFs accurately. The accuracy to plot speech on the IPA chart confirmed that the average accuracy is over 70%.

Academic Significance and Societal Importance of the Research Achievements

グローバル化により語学に秀でた人材の育成が必要不可欠な中,本システムは発話訓練のための自主学習支援ツールとしての利用とその教育効果が期待できる.現在の語学教育は読むことと聴くことに偏重しており,近年は話すことの重要性が指摘されているが,話すことについてはマンツーマン教育以外によい方法がなかった.本システムはこうした状況を打破するに,大きな助けとなると期待できる.加えて,今回開発した発音学習システムは,多言語音声言語コーパスを利用することにより,容易に他の言語へ応用することが可能である.そのため,英語だけではなく留学生の日本語教育や構音障碍者の発話訓練にも応用できると考えられる.

Report

(5 results)
  • 2018 Annual Research Report   Final Research Report ( PDF )
  • 2017 Research-status Report
  • 2016 Research-status Report
  • 2015 Research-status Report
  • Research Products

    (34 results)

All 2019 2018 2017 2016 2015 Other

All Journal Article (3 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 2 results,  Open Access: 1 results,  Acknowledgement Compliant: 1 results) Presentation (26 results) (of which Int'l Joint Research: 10 results) Remarks (4 results) Patent(Industrial Property Rights) (1 results)

  • [Journal Article] Speech corpus development toward speech recognition for oldest-old Japanese2017

    • Author(s)
      入部 百合絵,北岡 教英
    • Journal Title

      THE JOURNAL OF THE ACOUSTICAL SOCIETY OF JAPAN

      Volume: 73 Issue: 5 Pages: 303-310

    • DOI

      10.20697/jasj.73.5_303

    • NAID

      130006191400

    • ISSN
      0369-4232, 2432-2040
    • Related Report
      2017 Research-status Report
  • [Journal Article] Investigation of DNN-Based Audio-Visual Speech Recognition2016

    • Author(s)
      Satoshi Tamura, Hiroshi Ninomiya, Norihide Kitaoka, Shin Osuga, Yurie Iribe, Kazuya Takeda, Satoru Hayamizu
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E99.D Issue: 10 Pages: 2444-2451

    • DOI

      10.1587/transinf.2016SLP0019

    • NAID

      130005598220

    • ISSN
      0916-8532, 1745-1361
    • Related Report
      2016 Research-status Report
    • Peer Reviewed
  • [Journal Article] Using Reversed Sequences and Grapheme Generation Rules to Extend the Feasibility of a Phoneme Transition Network-Based Grapheme-to-Phoneme Conversion2016

    • Author(s)
      Seng Kheang, Kouichi Katsurada, Yurie Iribe and Tsuneo Nitta
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E99.D Issue: 4 Pages: 1182-1192

    • DOI

      10.1587/transinf.2015EDP7349

    • NAID

      130005141390

    • ISSN
      0916-8532, 1745-1361
    • Related Report
      2016 Research-status Report 2015 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research / Acknowledgement Compliant
  • [Presentation] 音声対話破綻検出手法の開発に向けた音声対話データ収録システム2019

    • Author(s)
      梶並 和明,西村 良太,入部 百合絵,北岡 教英
    • Organizer
      日本音響学会2019年春季研究発表会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 対話システムにおける非言語情報を用いた対話破綻の検出2019

    • Author(s)
      秋水紫苑,入部百合絵,北岡教英
    • Organizer
      情報処理学会 第81回全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 方言を考慮した音声言語情報に基づく高齢者認知症傾向の検出2019

    • Author(s)
      梅澤舞菜,入部百合絵,北岡教英
    • Organizer
      情報処理学会 第81回全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 発音動作可視化を目的とした口腔形状の特徴量抽出2019

    • Author(s)
      梅村直人,入部百合絵
    • Organizer
      情報処理学会 第81回全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 調音可視化に向けた深層学習による音声からの口腔形状推定2019

    • Author(s)
      梅村直人,入部百合絵
    • Organizer
      電子情報通信学会 東海支部 卒業研究発表会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 方言の違いを考慮した対話音声からの認知症傾向検出2019

    • Author(s)
      梅澤舞菜,入部百合絵,北岡教英
    • Organizer
      電子情報通信学会 東海支部 卒業研究発表会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 発話時脳波を利用した音声言語情報の識別2018

    • Author(s)
      深井 健大郎,大村 英史,桂田 浩一,平田 里佳,入部 百合絵,新田 恒雄
    • Organizer
      第5回サイレント音声認識ワークショップ
    • Related Report
      2018 Annual Research Report
  • [Presentation] 発話時脳波のスペクトル成分を用いた音声言語情報の識別2018

    • Author(s)
      平田里佳, 深井健大郎, 入部百合絵, 桂田浩一, 新田恒雄
    • Organizer
      第16回情報学ワークショップ
    • Related Report
      2018 Annual Research Report
  • [Presentation] Construction of a Corpus for Elderly Japanese Speech Recognition2018

    • Author(s)
      Meiko Fukuda, Ryota Nishimura, Norihide Kitaoka, Hiromitsu Nishizaki, Yurie Iribe
    • Organizer
      IEEE GCCE 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Construction of a Corpus of Elderly Japanese Speech for Analysis and Recognition2018

    • Author(s)
      Norihide Kitaoka, Yurie Iribe, Hiromitsu Nishizaki
    • Organizer
      LREC2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 高齢者の対話音声から抽出した言語的・音響的特徴に基づく認知症傾向の判別2018

    • Author(s)
      川島 愛美,入部 百合絵,北岡 教英
    • Organizer
      電子情報通信学会東海支部 卒業研究発表会
    • Related Report
      2017 Research-status Report
  • [Presentation] 英語発話学習に向けた音素連結・脱落同化パターンの分析と検出2018

    • Author(s)
      平田 里佳,入部 百合絵,新田 恒雄
    • Organizer
      教育システム情報学会 東海支部学生研究発表会
    • Related Report
      2017 Research-status Report
  • [Presentation] 音響的特徴を用いた自然発話音声と演技音声の比較分析2018

    • Author(s)
      仲村渠 遥,入部 百合絵
    • Organizer
      日本音響学会2018春季研究発表会講演論文集
    • Related Report
      2017 Research-status Report
  • [Presentation] 高齢者の対話音声から抽出した言語的・音響的特徴に基づく認知症傾向の判別2018

    • Author(s)
      川島 愛美,入部 百合絵,北岡 教英
    • Organizer
      日本音響学会2018春季研究発表会講演論文集
    • Related Report
      2017 Research-status Report
  • [Presentation] 英語発話学習に向けた音素連結・脱落同化パターンの分析と検出2018

    • Author(s)
      平田 里佳,入部 百合絵,新田 恒雄
    • Organizer
      日本音響学会2018春季研究発表会講演論文集
    • Related Report
      2017 Research-status Report
  • [Presentation] A Human Machine Interface Framework for Autonomous Vehicle Control2017

    • Author(s)
      Takuma Nakagawa, Ryota Nishimura, Yurie Iribe, Yoshio Ishiguro, Shin Ohsuga, Norihide Kitaoka
    • Organizer
      IEEE GCCE 2017
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Speech corpus spoken by young-old, old-old and oldest-old Japanese2016

    • Author(s)
      Yurie Iribe, Norihide Kitaoka, Shuhei Segawa
    • Organizer
      LREC2016
    • Place of Presentation
      Portoroz(Slovenia)
    • Year and Date
      2016-05-23
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Suffix Array を用いた高速STD システムにおけるリスコアリング法の検討2016

    • Author(s)
      石原 元気,桂田 浩一,新田 恒雄,入部 百合絵
    • Organizer
      日本音響学会2016年春季研究発表会講演論文集
    • Place of Presentation
      桐蔭横浜大学(神奈川県横浜市)
    • Year and Date
      2016-03-09
    • Related Report
      2015 Research-status Report
  • [Presentation] オートエンコーダと話者性変換ユニットを用いた声質変換法の提案2016

    • Author(s)
      入澤 浩太郎,桂田 浩一,新田 恒雄,入部 百合絵
    • Organizer
      日本音響学会2016年春季研究発表会講演論文集
    • Place of Presentation
      桐蔭横浜大学(神奈川県横浜市)
    • Year and Date
      2016-03-09
    • Related Report
      2015 Research-status Report
  • [Presentation] Audio-visual speech recognition using deep bottleneck features and high-performance lipreading2015

    • Author(s)
      Satoshi Tamura, Hiroshi Ninomiya, Norihide Kitaoka, Shin Osuga, Yurie Iribe, Kazuya Takeda
    • Organizer
      APSIPA ASC 2015
    • Place of Presentation
      香港(中国)
    • Year and Date
      2015-12-16
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] DEVELOPMENT OF NEW SPEECH CORPUS FOR ELDERLY JAPANESE SPEECH RECOGNITION2015

    • Author(s)
      Yurie Iribe, Norihide Kitaoka, Shuhei Segawa
    • Organizer
      O-COCOSDA 2015
    • Place of Presentation
      上海(中国)
    • Year and Date
      2015-10-28
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] Audio-visual processing toward robust speech recognition in cars2015

    • Author(s)
      Satoshi Tamura, Hiroshi Ninomiya, Norihide Kitaoka, Shin Osuga, Yurie Iribe, Kazuya Takeda, Satoru Hayami
    • Organizer
      DSP in vehicle 2015
    • Place of Presentation
      バークレー(アメリカ)
    • Year and Date
      2015-10-14
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] INVESTIGATION OF DNN-BASED MODELING FOR AUDIO-VISUAL SPEECH RECOGNITION2015

    • Author(s)
      Satoshi Tamura, Hiroshi Ninomiya, Norihide Kitaoka, Shin Osuga, Yurie Iribe, Kazuya Takeda, Satoru Hayamizu
    • Organizer
      MLSLP 2015
    • Place of Presentation
      会津大学(福島県会津若松市)
    • Year and Date
      2015-09-19
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] AutoEncoderを用いたActive Appearance Modelsの性能評価2015

    • Author(s)
      渡辺 拓也,桂田 浩一,新田 恒雄,入部 百合絵
    • Organizer
      電子情報通信学会技術研究報告PRMU2015-85
    • Place of Presentation
      愛媛大学(愛媛県松山市)
    • Year and Date
      2015-09-14
    • Related Report
      2015 Research-status Report
  • [Presentation] Integration of Deep Bottleneck Features for Audio-Visual Speech Recognition2015

    • Author(s)
      Hiroshi Ninomiya, Norihide Kitaoka, Satoshi Tamura, Yurie Iribe and Kazuya Takeda
    • Organizer
      InterSpeech 2015
    • Place of Presentation
      ドレスデン(ドイツ)
    • Year and Date
      2015-09-06
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] Model Prioritization Voting Schemes for Phoneme Transition Network-based Grapheme-to-Phoneme Conversion2015

    • Author(s)
      Seng Kheang, Kouichi Katsurada, Yurie Iribe and Tsuneo Nitta
    • Organizer
      CIST2015
    • Place of Presentation
      オタワ(カナダ)
    • Year and Date
      2015-05-11
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Remarks] http://www.ist.aichi-pu.ac.jp/~iribe/

    • Related Report
      2018 Annual Research Report
  • [Remarks] http://www.ist.aichi-pu.ac.jp/~iribe/

    • Related Report
      2017 Research-status Report
  • [Remarks]

    • URL

      http://www.ist.aichi-pu.ac.jp/~iribe/

    • Related Report
      2016 Research-status Report
  • [Remarks]

    • URL

      http://www.ist.aichi-pu.ac.jp/~iribe/

    • Related Report
      2015 Research-status Report
  • [Patent(Industrial Property Rights)] 発音動作可視化装置および発音学習装置2015

    • Inventor(s)
      入部百合絵,新田恒雄
    • Industrial Property Rights Holder
      国立大学法人 豊橋技術科学大学
    • Industrial Property Rights Type
      特許
    • Filing Date
      2015
    • Acquisition Date
      2017
    • Related Report
      2017 Research-status Report

URL: 

Published: 2015-04-16   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi