Study on speech synthesis for humanoid spoken dialog system

• Project/Area Number: 21800020
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Single-year Grants
• Research Field: Perception information processing/Intelligent robotics
• Research Institution: Tokyo Institute of Technology
• Principal Investigator: NOSE Takashi Tokyo Institute of Technology, 大学院・総合理工学研究科, 助教 (90550591)
• Project Period (FY): 2009 – 2010
• Project Status: Completed (Fiscal Year 2010)
• Budget Amount: ¥2,639,000 (Direct Cost: ¥2,030,000、Indirect Cost: ¥609,000); Fiscal Year 2010: ¥1,261,000 (Direct Cost: ¥970,000、Indirect Cost: ¥291,000); Fiscal Year 2009: ¥1,378,000 (Direct Cost: ¥1,060,000、Indirect Cost: ¥318,000)
• Keywords: テキスト音声合成 / 隠れマルコフモデル / 話し言葉音声 / 話者適応 / HMM音声合成 / ヒューマノイドロボット / 音声対話システム / 声質変換 / 感情音声 / ロバスト音声認識

Research Abstract

Two novel techniques and an investigation were presented that is key technologies of speech synthesis for the development of humanoid spoken dialog system as follows. (1) Spontaneous speech synthesis based on statistical parametric modeling (2) Speaker-independent voice conversion based on statistical parametric modeling. (3) Investigation of phonetic and prosodic contextual factors in speech synthesis.

Research Products

• [Journal Article] HMM-based voice conversion using quantized F0 context (2010)
  • Author(s): Takashi Nose, Yuhei Ota, Takao Kobayashi
  • Journal Title: IEICE Trans.on Information and Systems D vol.E93-9 Pages: 2483-2490
  • NAID: 10027640446
  • Peer Reviewed
• [Journal Article] Evaluation of prosodic contextual factors for HMM-based speech synthesis (2010)
  • Author(s): Shuji Yokomizo, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010 Pages: 430-433
  • Peer Reviewed
• [Journal Article] Conversational spontaneous speech synthesis using average voice model (2010)
  • Author(s): Tomoki Koriyama, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010 Pages: 853-856
  • Peer Reviewed
• [Journal Article] Speaker-independent HMM-based voice conversion using quantized fundamental frequency (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010 Pages: 1724-1727
  • Peer Reviewed
• [Journal Article] HMM-based robust voice conversion using adaptive F0 quantization (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.7th ISCA Workshop on Speech Synthesis, SSW7-2010 Pages: 80-85
  • Peer Reviewed
• [Journal Article] A rapid model adaptation technique for emotional speech recognition with style estimation based on multiple-regression HMM (2010)
  • Author(s): Yusuke Ijima, Takashi Nose, Makoto Tachibana, Takao Kobayashi
  • Journal Title: IEICE Trans.on Information and Systems D vol.E93-1 Pages: 107-115
  • NAID: 10026813194
  • Peer Reviewed
• [Journal Article] A technique for estimating intensity of emotional expressions and speaking styles in speech based on multiple-regression HSMM (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Journal Title: IEICE Trans.on Information and Systems D vol.E93-1 Pages: 116-124
  • NAID: 10026813214
  • Peer Reviewed
• [Journal Article] HMM-based voice conversion using quantized FO context (2010)
  • Author(s): Takashi Nose, Yuhei Ota, Takao Kobayashi
  • Journal Title: MICE Trans.on Information and Systems Volume: vol.E93-D,No.9 Pages: 2483-2490
  • Peer Reviewed
• [Journal Article] Evaluation of prosodic contextual factors for HMM-based speech synthes is (2010)
  • Author(s): Shuji Yokomizo, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.11th Annual Conference of the International Speech Communication Association Pages: 430-433
  • Peer Reviewed
• [Journal Article] Conversational spontaneous speech synthesis using average voice model (2010)
  • Author(s): Tomoki Koriyama, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.11th Annual Conference of the International Speech Communication Association Pages: 853-856
  • Peer Reviewed
• [Journal Article] Speaker-independent HMM-based voice conversion using quantized fund amental frequency (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.11th Annual Conference of the International Speech Communication Association Pages: 1724-1727
  • Peer Reviewed
• [Journal Article] HMM-based robust voice conversion using adaptive FO quantization (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.7th ISCA Workshop on Speech Synthesis Pages: 80-85
  • Peer Reviewed
• [Journal Article] A rapid model adaptation technique for emotional speech recognition with stylestimation based on multiple-regression HMM (2010)
  • Author(s): Yusuke Ijima, Takashi Nose, Makoto Tachibana, Takao Kobayashi
  • Journal Title: IEICE Trans. on Information and Systems Vol.E93-D, No. Pages: 107-115
  • Peer Reviewed
• [Journal Article] A technique for estimating intensity of emotional expressions and speaking styles in speech based on multiple-regression HSMM (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Journal Title: IEICE Trans. on Information and Systems Vol.E93-D, No. Pages: 116-124
  • NAID: 10026813214
  • Peer Reviewed
• [Journal Article] HMM-based speech synthesis with unsupervised labeling of accentual context based on FO quantization and average voice model (2010)
  • Author(s): Takashi Nose, Koujirou Ooki, Takao Kobayashi
  • Journal Title: Proc. 2010 IEEE International Conference on Acoustics, Speech and Signal Processing Pages: 4622-4625
  • Peer Reviewed
• [Journal Article] HMM-based speech synthesis with unsupervised labeling of accentual context based on F0 quantization and average voice model (2009)
  • Author(s): Takashi Nose, Koujirou Ooki, Takao Kobayashi
  • Journal Title: Proc.2010 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2010 Pages: 4622-4625
  • Peer Reviewed
• [Journal Article] Speaking style adaptation for spontaneous speech recognition using multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Takeshi Matsubara, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009 Pages: 552-555
  • Peer Reviewed
• [Journal Article] HMM-based speaker characteristics emphasis using average voice model (2009)
  • Author(s): Takashi Nose, Junichi Asada, Takao Kobayashi
  • Journal Title: Proc.10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009 Pages: 2631-2634
  • Peer Reviewed
• [Journal Article] A robust speaker-adaptive HMM-based text-to-speech synthesis (2009)
  • Author(s): Junichi Yamagishi, Takashi Nose, Heiga Zen, Zhenhua Ling, Tomoki Toda, Keiichi Tokuda, Simon King, Steve Renals
  • Journal Title: IEEE Trans.on Audio, Speech, and Language Processing vol.17, 6 Pages: 1208-1230
  • Peer Reviewed
• [Journal Article] Emotional speech recognition based on style estimation and adaptation with multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Makoto Tachibana, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc.2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009 Pages: 4157-4160
  • Peer Reviewed
• [Journal Article] Emotional speech recognition based on style estimation and adaptationwith multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Makoto Tachibana, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc. 2009 IEEE International Conference on Acoustics, Speech and Signal Processing Pages: 4157-4160
  • Peer Reviewed
• [Journal Article] A robust speaker-adaptive HMM-based text-to-speech synthesis (2009)
  • Author(s): Junichi Yamagishi, Takashi Nose, HeigaZen, Zhen-Hua Ling, Tomoki Toda, Keiichi Tokuda, Simon King, Steve Renals
  • Journal Title: IEEE Trans. on Audio, Speech, and Language Processing Vol.17, No.6 Pages: 1208-1230
  • Peer Reviewed
• [Journal Article] Speaking style adaptation for spontaneous speech recognition using multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Takeshi Matsubara, Takashi Nose, Takao Kobayashi
  • Journal Title: Proc. 10th Annual Conference of the International Speech Communication Association Pages: 552-555
  • Peer Reviewed
• [Journal Article] HMM-based speaker characteristics emphasis using average voice model (2009)
  • Author(s): Takashi Nose, Junichi Asada, Takao Kobayashi
  • Journal Title: Proc. 10th Annual Conference of the International Speech Communication Association Pages: 2631-2634
  • Peer Reviewed
• [Presentation] 日本語話し言葉コーパスを用いた対話音声合成のための音韻・韻律コンテキストの検討 (2011)
  • Author(s): 郡山知樹, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2011年春季研究発表会
  • Place of Presentation: 早稲田大学, 東京都新宿区
  • Year and Date: 2011-03-11
• [Presentation] 多様な発話様式によるHMM音声合成のための韻律コンテキストの検討 (2011)
  • Author(s): 前野悠, 能勢隆, 小林隆夫, 井島勇祐, 中嶋秀治, 水野秀之, 吉岡理
  • Organizer: 日本音響学会2011年春季研究発表会
  • Place of Presentation: 早稲田大学, 東京都新宿区
  • Year and Date: 2011-03-09
• [Presentation] 合成音声を用いた非パラレルデータによる声質変換の検討 (2011)
  • Author(s): 史潤宇, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2011年春季研究発表会
  • Place of Presentation: 早稲田大学, 東京都新宿区
  • Year and Date: 2011-03-09
• [Presentation] Speaker-independent HMM-based voice conversion using quantized fundamental frequency (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Organizer: Proc.11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
  • Place of Presentation: Makuhari, Japan.
  • Year and Date: 2010-09-29
• [Presentation] Speaker-independent HMM-based voice conversion using quantized fund amental frequency (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Organizer: 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
  • Place of Presentation: Makuhari, Japan
  • Year and Date: 2010-09-29
• [Presentation] Conversational spontaneous speech synthesis using average voice model (2010)
  • Author(s): Tomoki Koriyama, Takashi Nose, Takao Kobayashi
  • Organizer: Proc.11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
  • Place of Presentation: Makuhari, Japan.
  • Year and Date: 2010-09-28
• [Presentation] Conversational spontaneous speech synthesis using average voice model (2010)
  • Author(s): Tomoki Koriyama, Takashi Nose, Takao Kobayashi
  • Organizer: 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
  • Place of Presentation: Makuhari, Japan
  • Year and Date: 2010-09-28
• [Presentation] Evaluation of prosodic contextual factors for HMM-based speech synthesis (2010)
  • Author(s): Shuji Yokomizo, Takashi Nose, Takao Kobayashi
  • Organizer: Proc.11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
  • Place of Presentation: Makuhari, Japan.
  • Year and Date: 2010-09-27
• [Presentation] Evaluation of prosodic contextual factors for HMM-based speech synthes is (2010)
  • Author(s): Shuji Yokomizo, Takashi Nose, Takao Kobayashi
  • Organizer: 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
  • Place of Presentation: Makuhari, Japan
  • Year and Date: 2010-09-27
• [Presentation] HMM-based robust voice conversion using adaptive FO quantization (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Organizer: 7th ISCA Workshop on Speech Synthesis, SSW7
  • Place of Presentation: Kyoto, Japan
  • Year and Date: 2010-09-27
• [Presentation] HMM-based robust voice conversion using adaptive F0 quantization (2010)
  • Author(s): Takashi Nose, Takao Kobayashi
  • Organizer: Proc.7th ISCA Workshop on Speech Synthesis, SSW7-2010
  • Place of Presentation: Kyoto, Japan.
  • Year and Date: 2010-09-22
• [Presentation] HMMに基づく英語音声合成の韻律コンテキストの評価 (2010)
  • Author(s): 横溝秀始, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年秋季研究発表会
  • Place of Presentation: 関西大学, 大阪府吹田市
  • Year and Date: 2010-09-16
• [Presentation] 話者適応を用いたHMMに基づく不特定話者間声質変換 (2010)
  • Author(s): 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年秋季研究発表会
  • Place of Presentation: 関西大学, 大阪府吹田市
  • Year and Date: 2010-09-16
• [Presentation] 適応FO量子化によるHMM声質変換の品質改善 (2010)
  • Author(s): 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年秋季研究発表会
  • Place of Presentation: 関西大学, 大阪府吹田市
  • Year and Date: 2010-09-16
• [Presentation] 二段階モデル適応に基づく対話音声合成の検討 (2010)
  • Author(s): 郡山知樹, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年秋季研究発表会
  • Place of Presentation: 関西大学, 大阪府吹田市
  • Year and Date: 2010-09-15
• [Presentation] HMM-based speech synthesis with unsupervised labeling of accentual context based on F0 quantization and average voice model (2010)
  • Author(s): Takashi Nose, Koujirou Ooki, Takao Kobayashi
  • Organizer: 2010 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2010
  • Place of Presentation: Dallas, USA.
  • Year and Date: 2010-03-17
• [Presentation] HMM-based speech synthesis with unsupervised labeling of accentual context based on FO quantization and average voice model (2010)
  • Author(s): Takashi Nose, Koujirou Ooki, Takao Kobayashi
  • Organizer: 2010 IEEE Interantional Conference on Acoustics, Speech and Signal Processing, ICASSP 2010
  • Place of Presentation: Dallas, Texas, USA
  • Year and Date: 2010-03-17
• [Presentation] HMMに基づく対話音声合成のための発話単位の検討 (2010)
  • Author(s): 郡山知樹, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年春季研究発表会
  • Place of Presentation: 電気通信大学,東京都調布市
  • Year and Date: 2010-03-10
• [Presentation] 量子化FO韻律コンテキストを用いたHMM音声合成の評価 (2010)
  • Author(s): 大木康次郎, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年春季研究発表会
  • Place of Presentation: 電気通信大学,東京都調布市
  • Year and Date: 2010-03-09
• [Presentation] HMM音声合成における韻律コンテキストの評価 (2010)
  • Author(s): 横溝秀始, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2010年春季研究発表会
  • Place of Presentation: 電気通信大学,東京都調布市
  • Year and Date: 2010-03-08
• [Presentation] 平均声に基づく対話音声合成に関する検討 (2010)
  • Author(s): 郡山知樹, 能勢隆, 小林隆夫
  • Organizer: 電子情報通信学会・音声研究会
  • Place of Presentation: 京都大学,京都市
  • Year and Date: 2010-01-21
• [Presentation] FO量子化に基づく韻律コンテキストを用いたHMM音声合成 (2009)
  • Author(s): 大木康次郎, 能勢隆, 小林隆夫
  • Organizer: 電子情報通信学会
  • Place of Presentation: 東京大学,東京都文京区
  • Year and Date: 2009-12-21
• [Presentation] HMMに基づく対話音声合成の検討 (2009)
  • Author(s): 郡山知樹, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2009年秋季研究発表会
  • Place of Presentation: 日本大学,福島県郡山市
  • Year and Date: 2009-09-15
• [Presentation] HMM音声合成におけるFOモデルの教師なし学習の検討 (2009)
  • Author(s): 大木康次郎, 能勢隆, 小林隆夫
  • Organizer: 日本音響学会2009年秋季研究発表会
  • Place of Presentation: 日本大学,福島県郡山市
  • Year and Date: 2009-09-15
• [Presentation] HMM-based speaker characteristics emphasis using average voice model (2009)
  • Author(s): Takashi Nose, Junichi Asada, Takao Kobayashi
  • Organizer: Proc.10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009
  • Place of Presentation: Brighton, U.K.
  • Year and Date: 2009-09-10
• [Presentation] HMM-based speaker characteristics emphasis using average voice model (2009)
  • Author(s): Takashi Nose, Junichi Asada, Takao Kobayashi
  • Organizer: 10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009
  • Place of Presentation: Brighton, UK
  • Year and Date: 2009-09-10
• [Presentation] Speaking style adaptation for spontaneous speech recognition using multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Takeshi Matsubara, Takashi Nose, Takao Kobayashi
  • Organizer: Proc.10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009
  • Place of Presentation: Brighton, U.K.
  • Year and Date: 2009-09-07
• [Presentation] Speaking style adaptation for spontaneous speech recognition using multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Matsubara, Takashi Nose, Takao Kobayashi
  • Organizer: 10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009
  • Place of Presentation: Brighton, UK
  • Year and Date: 2009-09-07
• [Presentation] 重回帰HMMに基づく自然発話音声の発話様式識別 (2009)
  • Author(s): 能勢隆, 松原健, 井島勇祐, 小林隆夫
  • Organizer: 電子情報通信学会 音声研究会
  • Place of Presentation: 飯坂ホテル聚楽,福島県福島市
  • Year and Date: 2009-07-18
• [Presentation] Emotional speech recognition based on style estimation and adaptation with multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Makoto Tachibana, Takashi Nose, Takao Kobayashi
  • Organizer: Proc.2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009, 4157-4160
  • Place of Presentation: Taipei, Taiwan.
  • Year and Date: 2009-04-21
• [Presentation] Emotional speech recognition based on style estimation and adaptationwith multiple-regression HMM (2009)
  • Author(s): Yusuke Ijima, Makoto Tachibana, Takashi Nose, Takao Kobayashi
  • Organizer: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2009
  • Place of Presentation: Taipei, Taiwan
  • Year and Date: 2009-04-21