The development of a talking robot that estimates speech articulation from voice and the proposal of an interactive speech training system

• Project/Area Number: 18500152
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Perception information processing/Intelligent robotics
• Research Institution: Kagawa University
• Principal Investigator: SAWADA Hideyuki Kagawa University, 工学部, 准教授 (00308206)
• Project Period (FY): 2006 – 2008
• Project Status: Completed (Fiscal Year 2008)
• Budget Amount: ¥4,200,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥600,000); Fiscal Year 2008: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000); Fiscal Year 2007: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2006: ¥1,600,000 (Direct Cost: ¥1,600,000)
• Keywords: 発話ロボット / 音声 / 聴覚フィードバック / 自己組織化ニューラルネットワーク / 適応学習 / 聴覚障がい者 / 発話訓練 / 支援技術 / 自己組織化ニューラルネット / 聴覚障碍者

Research Abstract

人間の発声器官である肺、気道、声帯、声道、鼻腔、舌とそれらを動かす為の筋肉などを全て機械系により再現し、自律的な発話動作によって音声を生成する発話ロボットを構築した。本発話ロボットが、音声の特徴から、その音声を生成するのに必要な口内の発話動作が再現できることを確認した。更に、聴覚障碍者や発話障碍者が、ロボットの見本となる構音動作や口内の動きを見ながら、対話的に発話訓練をおこなうシステムを構築し、実証実験によりその有効性を確認した。

Research Products

• [Journal Article] A Robotic Voice Simulator and the Interactive Training for Hearing-Impaired People (2008)
  • Author(s): Hideyuki Sawada, Mitsuki Kitani and Yasumori Hayashi
  • Journal Title: Journal of Biomedicine and Biotechnology Volume 2008, Article ID 768232
  • Peer Reviewed
• [Journal Article] Interactive training of speech articulation for hearing impaired using a talking robot (2008)
  • Author(s): Mitsuki Kitani, Yasumori Hayashi and Hideyuki Sawada
  • Journal Title: International Conference on Disability, Virtual Reality and Associated Technologies ICDVRAT2008 Pages: 293-301
  • Peer Reviewed
• [Journal Article] A Robotic Voice Simulator and the Articulatory Reproduction of Impaired Voices (2008)
  • Author(s): Mitsuki Kitani, Yasumori Hayashi, and Hideyuki Sawada
  • Journal Title: Mecatronics2008 Pages: 203-203
  • Peer Reviewed
• [Journal Article] Sensing of Particular Speakers for the Construction of Voice Interface Utilized in Noisy Environment (2008)
  • Author(s): Hideyuki Sawada and Minoru Ohkado
  • Journal Title: Journal of Electrical Engineering in Japan Vol.162, No.3 Pages: 78-86
  • NAID: 10018318536
  • Peer Reviewed
• [Journal Article] A Robotic Voice Simulator and the Interactive Training for Hearing-Impaired People (2008)
  • Author(s): Hideyuki Sawada, Mitsuki Kitani, Yasumori Hayashi
  • Journal Title: Journal of Biomedicine and Biotechnology Volume 2008
  • Peer Reviewed
• [Journal Article] Interactive training of speech articulation for hearing impaired using a talking robot (2008)
  • Author(s): Mitsuki Kitani, Yasumori Hayashi, Hideyuki Sawada
  • Journal Title: International Conference on Disability, Virtual Reality and Associated Technologies Pages: 293-301
  • Peer Reviewed
• [Journal Article] A Robotic Voice Simulator and the Articulatory Reproduction of Impaired Voices (2008)
  • Author(s): Mitsuki Kitani, Yasumori Hayashi, Hideyuki Sawada
  • Journal Title: Mecatronics2008 Pages: 203-203
  • Peer Reviewed
• [Journal Article] "'Sensing of Particular Speakers for the Construction of Voice Interface Utilized in Noisy Environment" (2008)
  • Author(s): Hideyuki Sawada and Minoru Ohkado
  • Journal Title: Journal of Electrical Engineering in Japan Vol.162, No.3 Pages: 78-86
  • NAID: 10018318536
  • Peer Reviewed
• [Journal Article] ヒューマンコミュニケーション賞受賞(電子情報通信学会 ヒューマンコミュニケーショングループ) (2007)
  • Author(s): 澤田秀之、中村光宏、林恭守、木谷光来
  • Journal Title: 「発話ロボットを用いた聴覚障碍者のための発話訓練」, 電子情報通信学会技術研究報告 HCS2007-24 Pages: 125-130
• [Journal Article] New Technology Foundation Award Finalist (2007)
  • Author(s): Hideyuki Sawada and Mitsuhiro Nakamura
  • Journal Title: "Mechanical Voice System and its Singing Performance", International Conference on Intelligent Robots and Systems (IROS2007) Proc. of IROS2004 Pages: 1920-1925
  • Peer Reviewed
• [Journal Article] A Robotic Auditory System that Interacts with Musical Sounds and Human Voices (2007)
  • Author(s): Hideyuki Sawada and Toshiya Takechi
  • Journal Title: Journal of Advanced Computational Intelligence and Intelligent Informatics Vol.11, No.10 Pages: 1177-1183
  • Peer Reviewed
• [Journal Article] Articulatory Reproduction of Voices of Hearing-impaired by a Talking Robot (2007)
  • Author(s): Hideyuki Sawada, Mitsuki Kitani and Yasumori Hayashi
  • Journal Title: Frontiers in the Convergence of Bioscience and Information Technologies (FBIT2007) Pages: 603-608
  • Peer Reviewed
• [Journal Article] A Robotic Mechanical Voice Simulator to Train Auditory Impaired People (2007)
  • Author(s): Hideyuki Sawada, Yasumori Hayashi and Mitsuki Kitani
  • Journal Title: Malaysia Japan International Symposium on Advanced Technology (MJISAT2007) MJISAT- 319_SS
  • Peer Reviewed
• [Journal Article] A Talking Robot and the Reproduction of Human Voice (2007)
  • Author(s): Mitsuki Kitan, Yasumori Hayashi and Hideyuki Sawada
  • Journal Title: 14th Tri-University International Joint Seminar and Symposium Pages: 347-352
  • Peer Reviewed
• [Journal Article] Estimation of musical pitch by using comb filters for the identification of musical instruments (2007)
  • Author(s): Atsushi Todo and Hideyuki Sawada
  • Journal Title: SICE Annual Conference 2007 Pages: 660-663
  • Peer Reviewed
• [Journal Article] ヒューマンコミュニケーション賞受賞(2008年3月23日)「発話ロボットを用いた聴覚障碍者のための発話訓練」 (2007)
  • Author(s): 澤田 秀之、中村 光宏、林 恭守、木谷 光来
  • Journal Title: 電子情報通信学会 技術研究報告 ヒューマンコミュニケーション基礎 HCS2007-24 Pages: 125-130
  • Peer Reviewed
• [Journal Article] New Technology Foundation Award Finalist(2007年11月2日)"Mechanical Voice System and its Singing Performance" (2007)
  • Author(s): Hideyuki Sawada and Mitsuhiro Nakamura
  • Journal Title: International Conference on Intelligent Robots and Systems(IROS2007) IROS2004 Pages: 1920-1925
  • Peer Reviewed
• [Journal Article] "A Robotic Auditory System that Interacts with Musical Sounds and Human Voices" (2007)
  • Author(s): Hideyuki Sawada and Toshiya Takechi
  • Journal Title: Journal of Advanced Computational Intelligence and Intelligent Informatics Vol.11, No.10 Pages: 1177-1183
  • Peer Reviewed
• [Journal Article] "Articulatory Reproduction ofVoices of Hearing-impaired by a Talking Robot" (2007)
  • Author(s): Hideyuki Sawada, Mitsuki Kitani and Yasumori Hayashi
  • Journal Title: Frontiers in the Convergence of Bioscience and Information Technologies FBIT2007 Pages: 603-608
  • Peer Reviewed
• [Journal Article] "A Robotic Mechanical Voice Simulator to Train Auditory Impaired People" (2007)
  • Author(s): Hideyuki Sawada, Yasumori Hayashi and Mitsuki Kitani
  • Journal Title: Malaysia Japan International Symposium on Advanced Technology MJISAT2007
  • Peer Reviewed
• [Journal Article] "A Talking Robot and the Reproduction of Human Voice" (2007)
  • Author(s): Mitsuki Kitan, Yasumori Hayashi and Hideyuki Sawada
  • Journal Title: 14th Tri-University International Joint Seminar and Symposium Pages: 347-352
  • Peer Reviewed
• [Journal Article] "Estimation ofmusical pitch by using comb filters for the identification of musical instruments" (2007)
  • Author(s): Atsushi Todo and Hideyuki Sawada
  • Journal Title: Proceedings of SICE Annual Conference 2007 Pages: 660-663
  • Peer Reviewed
• [Journal Article] 雑音環境下における音声インタフェース構築のための特定話者のセンシング (2006)
  • Author(s): 澤田秀之、大加戸稔
  • Journal Title: 電気学会論文誌D Vol.126, No.11 Pages: 1446-1453
  • NAID: 10018318536
  • Peer Reviewed
• [Journal Article] Talking Robot and the Analysis of Autonomous Voice Acquisition (2006)
  • Author(s): Mitsuhiro Nakamura and Hideyuki Sawada
  • Journal Title: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2006) Pages: 4684-4689
  • Peer Reviewed
• [Journal Article] Realtime Speaker Tracking for Robotic Auditory System (2006)
  • Author(s): Hideyuki Sawada, Atsushi Todo and Toshiya Takechi
  • Journal Title: Annual Conference of the IEEE Industrial Electronics Society (IECON2006) Pages: 5474-5479
  • Peer Reviewed
• [Journal Article] 雑音環境下における音声インターフェース構築のための特定話者のセンシング (2006)
  • Author(s): 澤田秀之, 大加戸稔
  • Journal Title: 電気学会論文誌D Vol.126 No.11 Pages: 1446-1453
  • NAID: 10018318536
• [Journal Article] Talking Robot and the Analysis of Autonomous Voice Acquisition (2006)
  • Author(s): Mitsuhiro Nakamura, Hideyuki Sawada
  • Journal Title: IEEE/RSJ International Conference on Intelligent Robots and Systems Pages: 4684-4689
• [Journal Article] Realtime Speaker Tracking for Robotic Auditory System (2006)
  • Author(s): Hideyuki Sawada, Atsushi Todo, Toshiya Takechi
  • Journal Title: Annual Conference of the IEEE Industrial Electronics Society Pages: 5474-5479
• [Journal Article] ベストプレゼンテーション章 受賞 発話ロボットの発話動作獲得と歌声の生成 (2006)
  • Author(s): 中村光宏, 林恭守, 澤田秀之
  • Journal Title: 情報処理学会 研究報告(情報処理学会 音楽情報科学研究会 夏のシンポジウム) 2006-MUS-66 Pages: 135-140
• [Journal Article] IEEE Presentation Award 受賞 Voice Acquisition of a Talking Robot Using a Self-organizing Map (2006)
  • Author(s): Mitsuhiro Nakamura, Hideyuki Sawada
  • Journal Title: 電気関係学会 四国支部大会 講演論文集 Pages: 343-343
• [Journal Article] Acoustic analysis of esophageal speech (2006)
  • Author(s): Takuma Ise, Hideyuki Sawada
  • Journal Title: 電気関係学会 四国支部大会 講演論文集 Pages: 344-344
• [Presentation] A Talking Robot and the Interactive Speech Training for Hearing-Impaired (2009)
  • Author(s): Invited Talk, Hideyuki Sawada
  • Organizer: General Meeting of the Acoustical Society of America
  • Place of Presentation: Portland, USA
  • Year and Date: 2009-05-18
• [Presentation] ロボットアームを用いた音場の能動センシング (2008)
  • Author(s): 東藤篤史、澤田秀之
  • Organizer: 計測自動制御学会第9回システムインテグレーション部門講演会(SI2008)
  • Place of Presentation: 岐阜 長良川国際会議場
  • Year and Date: 2008-12-06
• [Presentation] ロボットアームを用いた音場の能動センシング (2008)
  • Author(s): 東藤篤史, 澤田秀之
  • Organizer: 計測自動制御学会第9回システムインテグレーション部門講演会(SI2008)
  • Place of Presentation: 岐阜
  • Year and Date: 2008-12-06
• [Presentation] Active Tracking of Particular Person Using Visual and Auditory Information (2008)
  • Author(s): Hideyuki Sawada
  • Organizer: International Symposium for the Establishment of Sustainable and Recycling-based Society
  • Place of Presentation: 香川大学
  • Year and Date: 2008-10-16
• [Presentation] IEEE Best Presentation Award 受賞 (2008)
  • Author(s): Mitsuki Kitani and Hideyuki Sawada
  • Organizer: "Proposal of 3D SOM for Autonomous Voice Acquisition of a Talking Robot", 電気関係学会四国支部大会講演論文集
  • Place of Presentation: 徳島大学
  • Year and Date: 2008-09-27
• [Presentation] Active sensing of sound location using a robotic arm (2008)
  • Author(s): Atsushi Todo and Hideyuki Sawada
  • Organizer: 電気関係学会四国支部大会講演論文集
  • Place of Presentation: 徳島大学
  • Year and Date: 2008-09-27
• [Presentation] IEEE Best Presentation Award, Proposal of 3D SOM for Autonomous Voice Acquisition of a Talking Robot (2008)
  • Author(s): Mitsuki Kitani, Hideyuki Sawada
  • Organizer: 電気関係学会四国支部大会講演論文集
  • Place of Presentation: 徳島大学
  • Year and Date: 2008-09-27
• [Presentation] Evolution of Talking Robot (2007)
  • Author(s): Invited Lecture, Hideyuki Sawada
  • Organizer: Universiti Teknologi Malaysia
  • Place of Presentation: マレーシア工科大学(マレーシア)
  • Year and Date: 2007-11-19
• [Presentation] Invited Lecture,"Evolution of Talking Robot" (2007)
  • Author(s): Hideyuki Sawada
  • Organizer: Universiti Telcnologi Malaysia
  • Place of Presentation: マレーシア工科大学
  • Year and Date: 2007-11-19
• [Presentation] IEEE Best Presentation Award 受賞 (2007)
  • Author(s): Yasumori Hayashi and Hideyuki Sawada
  • Organizer: "Analysis of Acquired Vocal Sounds by a Talking Robot" : 電気関係学会 四国支部大会 講演論文集
  • Place of Presentation: 徳島大学
  • Year and Date: 2007-09-29
• [Presentation] Extraction of voice characteristics for speech training of auditory impaired people (2007)
  • Author(s): Takuma Ise and Hideyuki Sawada
  • Organizer: 電気関係学会四国支部大会 : 講演論文集
  • Place of Presentation: 徳島大学
  • Year and Date: 2007-09-29
• [Presentation] IEEE Best Presentation Award"Analysis ofAcquired Vocal Sounds by a Talking Robot" (2007)
  • Author(s): Yasumori Hayashi and Hideyuki Sawada
  • Organizer: 電気関係学会四国支部大会
  • Place of Presentation: 徳島大学
  • Year and Date: 2007-09-29
• [Presentation] "Extraction of voice characteristics for speech training of auditory impaired people" (2007)
  • Author(s): Takuma Ise and Hideyuki Sawada
  • Organizer: 電気関係学会 四国支部大会
  • Place of Presentation: 徳島大学
  • Year and Date: 2007-09-29
• [Presentation] 発話ロボットによる発話動作の再現と肢体不自由児の発話訓練への応用 (2007)
  • Author(s): 澤田秀之
  • Organizer: 第52回西日本肢体不自由児施設運営研究会
  • Place of Presentation: 高松リーガホテルゼスト(招待講演)
  • Year and Date: 2007-09-14
• [Presentation] 招待講演「発話ロボットによる発話動作の再現と肢体不自由児の発話訓練への応用」 (2007)
  • Author(s): 澤田 秀之
  • Organizer: 第52回西日本肢体不自由児施設運営研究会
  • Place of Presentation: 高松リーガホテルゼスト
  • Year and Date: 2007-09-14
• [Presentation] IEEE Presentation Award 受賞Voice Acquisition of a Talking Robot Using a Self-organizing Map (2006)
  • Author(s): Mitsuhiro Nakamura and Hideyuki Sawada
  • Organizer: 電気関係学会 四国支部大会 : 講演論文集
  • Place of Presentation: 愛媛大学
  • Year and Date: 2006-09-26
• [Presentation] Acoustic analysis of esophageal speech (2006)
  • Author(s): Takuma Ise and Hideyuki Sawada
  • Organizer: 電気関係学会 四国支部大会 : 講演論文集
  • Place of Presentation: 愛媛大学
  • Year and Date: 2006-09-26
• [Presentation] ベストプレゼンテーション賞受賞「発話ロボットの発話動作獲得と歌声の生成」 (2006)
  • Author(s): 中村光宏、林恭守、澤田秀之
  • Organizer: 情報処理学会音楽情報科学研究会夏のシンポジウム : 情報処理学会研究報告
  • Place of Presentation: 香川ヴィラ塩江
  • Year and Date: 2006-08-08
• [Book] "Talking Robot and the Autonomous Acquisition of Talking Robot and the Autonomous Acquisition of Vocalization and Singing Skill", Chapter 22 in Robust Speech Recognition and Understanding, Edited by Grimm and Kroschel (2007)
  • Author(s): Hideyuki Sawada
  • Publisher: I-Tech Education and Publishing
• [Remarks]
  • URL: http://www.eng.kagawa-u.ac.jp/~sawada/
• [Remarks]
  • URL: http://www.eng.kagawa-u.ac.jp/~sawada/
• [Remarks]
  • URL: http://www.eng.kagawa-u.ac.jp/~sawada/