• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Research on singing rendering systems design based on an active auditory perception model

Research Project

Project/Area Number 14380165
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionWakayama University

Principal Investigator

KAWAHARA Hideki  Wakayama University, Department of Design Information Sciences, Professor, システム工学部, 教授 (40294300)

Co-Investigator(Kenkyū-buntansha) KATAYOSE Haruhiro  Kwansei Gakuin University, School of Science and Technology, Professor, 理工学部, 教授 (70294303)
NISHIURA Takanobu  Ritsumeikan University, College of Information Science and Engineering, Associate Professor, 情報理工学部, 助教授 (70343275)
BANNO Hideki  Wakayama University, Department of Design Information Sciences, Research Assistant, システム工学部, 助手 (20335003)
NISHIMURA Ryuichi  Wakayama University, Department of Design Information Sciences, Research Assistant, システム工学部, 助手 (00379611)
TAKAHASHI Toru  Wakayama University, Department of Design Information Sciences, Researcher, システム工学部, 研究員
Project Period (FY) 2002 – 2004
Project Status Completed (Fiscal Year 2004)
Budget Amount *help
¥14,300,000 (Direct Cost: ¥14,300,000)
Fiscal Year 2004: ¥4,200,000 (Direct Cost: ¥4,200,000)
Fiscal Year 2003: ¥6,300,000 (Direct Cost: ¥6,300,000)
Fiscal Year 2002: ¥3,800,000 (Direct Cost: ¥3,800,000)
KeywordsSpeech analysis / Speech synthesis / Auditory morphing / Fundamental frequency / Radiation pattern / Paralinguistic information / Singing synthesis / Speech dynamics / 高品質音声合成 / 歌唱システム / STRAIGHT / 音声変換 / ビブラート / 表情づけ / 非言語情報 / モーフィング / 韻律情報 / 聴覚フィードバック / 変換聴覚フィードバック
Research Abstract

The goal of this project is to investigate the source of reason why vocal music is attractive even without lyrics. This general goal was broken down to several sub-goal which consists of new research tool development and winning prize as the best artificial singing system at international contests. These goals were fulfilled even though the success introduced more questions than the answered questions. Firstly, the piece of chorus with artificially manipulated synthesized voices (that is an excerpt of a composition made by Toru Takemitsu titled "small sky") won the first prize among four synthetic singing systems at RENCON'04, the satellite event of the international conference on computer based entertainment systems (NIME'04) held in Shizuoka in 2004. The piece was made using a STRAIGHT based singing synthesis program. Secondly, the singing synthesis system is based on the auditory morphing algorithm invented for this research project. The morphing algorithm made a substantial impact … More on speech perception and music perception research and the algorithm is currently used in many research institutes worldwide. Thirdly, a new algorithm called "senza vibrato" was developed to made it possible to morph vibrato that is an essential ingredient of singing voice, and at the same time, is an obstacle that made morphing of singing voice very difficult. Fourthly, important experiences were obtained by performing actual investigations based on the "systematic downgrading strategy" that was proposed to characterize the current research project. Those accomplishments were reported at various international/domestic conferences and scientific journals. Those publications and the new research tools based on STRAIGHT made a research trend that is characterized by ecological views on auditory and speech perception. In conclusion, the project was a great success. However, it is important to note that even with all the accomplishments in this project, there still remains a huge gap between synthetic singers and human singers. There is a huge room for investigations to bridge this gap. The prospective research project may need to put attentions on methods for generalization from relatively small number of instances, because, based on experiences in this research project, it is generally impractical to provide sufficient number of singing voice instances to function the "systematic downgrading strategy" in its full extent. Less

Report

(4 results)
  • 2004 Annual Research Report   Final Research Report Summary
  • 2003 Annual Research Report
  • 2002 Annual Research Report
  • Research Products

    (57 results)

All 2005 2004 2003 2002 Other

All Journal Article (44 results) Book (1 results) Publications (12 results)

  • [Journal Article] The processing and perception of size information in speech sounds2005

    • Author(s)
      David R.R.Smith
    • Journal Title

      the Journal of the Acoustical Society of America 117・1

      Pages: 305-318

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Annual Research Report 2004 Final Research Report Summary
  • [Journal Article] Underlying principles of a high-quality speech manipulation system STRAIGHT and its application to speech segregation2005

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Speech separation by human and machines

      Pages: 167-180

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation using an event-synchronous auditory image and STRAIGHT2005

    • Author(s)
      Toshio Irino
    • Journal Title

      Speech separation by human and machines

      Pages: 151-162

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] The processing and perception of size information in speech sounds2005

    • Author(s)
      David R.R.Smith
    • Journal Title

      the Journal of the Acoustical Society of America Vol.117, No.1

      Pages: 305-318

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] "senza vibrato" : a key component for morphing singing2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.8th International Conference on Spoken Language Processing (ICSLP 2004) V

      Pages: 934-937

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Intelligibility of degraded speech from smeared STRAIGHT spectrum2004

    • Author(s)
      Jiang Jin
    • Journal Title

      Proc.8th International Conference on Spoken Language Processing (ICSLP 2004) IV

      Pages: 530-533

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Loudspeaker equalization based on multi-location observation with reliable time-frequency region selection and its evaluation using sound propagation measurement2004

    • Author(s)
      Masanori Morise
    • Journal Title

      Proc.12th European Signal Processing Conference (EUSIPCO 2004)

      Pages: 1995-1998

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Acappella synthesis demonstrations using RWC music database2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.International Conference on New Interfaces for Musical Expression (NIMEO4)

      Pages: 130-131

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Algorithm amalgam : Morphing waveform based methods, sinuisoidal models and STRAIGHT2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004) I

      Pages: 13-16

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Computational basis of illusionary pitch perception2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) II

      Pages: 1081-1084

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] A new acoustic measurement and compensation method based on logarithmic transformation of the time axis and multi-location acquisition2004

    • Author(s)
      Masanori Morise
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) I

      Pages: 721-724

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Effects of group delay diffusion in pulse trains on timbre : a periodicity cue in auditory images2004

    • Author(s)
      Minoru Tsuzaki
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) II

      Pages: 1803-1806

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation using an auditory vocoder with event-synchronous enhancement2004

    • Author(s)
      Toshio Irino
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) IV

      Pages: 3025-3028

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] "senza vibrato" : a key component for morphing singing2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.8th International Conference on Spoken Language Processing (ICSLP 2004) vol.V

      Pages: 934-937

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Intelligibility of degraded speech from smeared STRAIGHT spectrum2004

    • Author(s)
      Jiang Jin
    • Journal Title

      Proc.8th International Conference on Spoken Language Processing (ICSLP 2004) vol.IV

      Pages: 530-533

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Acappella synthesis demonstrations using RWC music database2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.International Conference on New Interfaces for Musical Expression (NIME04)

      Pages: 130-131

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Algorithm amalgam : Morphing waveform based methods, sinuisoidal models and STRAIGHT2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004) vol.I

      Pages: 13-16

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Computational basis of illusionary pitch perception2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) vol.II

      Pages: 1081-1084

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] A new acoustic measurement and compensation method based on logarithmic transformation of the time axis and multi-location acquisition2004

    • Author(s)
      Masanori Morise
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) vol.I

      Pages: 721-724

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Effects of group delay diffusion in pulse trains on timbre : a periodicity cue in auditory images2004

    • Author(s)
      Minoru Tsuzaki
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) vol.II

      Pages: 1803-1806

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation using an auditory vocoder with event-synchronous enhancement2004

    • Author(s)
      Toshio Irino
    • Journal Title

      Proc.18th International Congress on Acoustics (ICA 2004) vol.IV

      Pages: 3025-3028

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] "senza vibrato": a key component for morphing singing2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.ICSLP 2004 V

      Pages: 934-937

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Acappella synthesis demonstrations using RWC music database2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.NIME' 04 I

      Pages: 130-131

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Algorithm AMALGAN : Morphing waveform based methods, sinusoidal models and STRAIGHT2004

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.ICASSP2004 I

      Pages: 13-16

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 聴覚フィードバックの発声への影響-ヒトは自分の話声を聞いているのか?2003

    • Author(s)
      河原英紀
    • Journal Title

      日本音響学会誌 59・11

      Pages: 670-675

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] 発話時の頭部周辺での音声の伝達特性について2003

    • Author(s)
      貫名真澄
    • Journal Title

      日本音響学会誌 59・5

      Pages: 256-260

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Exemplar-based voice quality analysis and control using a high quality auditory morphing procedure based on STRAIGHT2003

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.Voice Quality : Functions, Analysis and Synthesis (VOQUAL'03), International Speech Communication Association Tutorial and Research Workshop

      Pages: 109-114

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation based on fundamental event information using an auditory VOCODER2003

    • Author(s)
      Toshio Irino
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech'03)

      Pages: 553-556

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Investigation of emotionally morphed speech perception and its structure using a high quality speech manipulation System2003

    • Author(s)
      Hisami Matsui
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech'03)

      Pages: 2113-2116

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Glottal closure instant synchronous sinusoidal model for high quality speech analysis/synthesis2003

    • Author(s)
      Parham Zolfaghari
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech'03)

      Pages: 2441-2444

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Influence of recording equipment on the identification of second language phoneme contrasts2003

    • Author(s)
      Hiroaki Kato
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech'03)

      Pages: 3157-3160

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Psychological evaluation of emotional speech using a new morphing method2003

    • Author(s)
      Yuko Sogabe
    • Journal Title

      Proc.4th International Conference on Cognitive Science (ICCS) 2

      Pages: 628-633

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Auditory morphing based on an elastic perceptual distance metric in an interference-free time-frequency representation2003

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003) I

      Pages: 256-259

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation using event synchronous auditory VOCODER2003

    • Author(s)
      Toshio Irino
    • Journal Title

      Proc.2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003) V

      Pages: 525-528

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Auditory feedback effects on speech production : Are humans listening to their own speech?2003

    • Author(s)
      Hideki Kawahara
    • Journal Title

      the Journal of the Acoustical Society of Japan Vol.59, No.11

      Pages: 670-675

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Transfer characteristics of speech sounds around speaker's head2003

    • Author(s)
      Masumi Nukina
    • Journal Title

      the Journal of the Acoustical Society of Japan Vol.59, No.5

      Pages: 256-260

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation based on fundamental event information using an auditory VOCODER2003

    • Author(s)
      Toshio Irino
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech '03)

      Pages: 553-556

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Investigation of emotionally morphed speech perception and its structure using a high quality speech manipulation System2003

    • Author(s)
      Hisami Matsui
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech '03)

      Pages: 2113-2116

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Influence of recording equipment on the identification of second language phoneme contrasts2003

    • Author(s)
      Hiroaki Kato
    • Journal Title

      Proc.8th European Conference on Speech Communication and Technology (Eurospeech '03)

      Pages: 3157-3160

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Psychological evaluation of emotional speech using a new morphing method2003

    • Author(s)
      Yuko Sogabe
    • Journal Title

      Proc.4th International Conference on Cognitive Science (ICCS) vol.2

      Pages: 628-633

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Auditory morphing based on an elastic perceptual distance metric in an interference-free time-frequency representation2003

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003) vol.I

      Pages: 256-259

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] Speech segregation using event synchronous auditory VOCODER2003

    • Author(s)
      Toshio Irino
    • Journal Title

      Proc.2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003) vol.V

      Pages: 525-528

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] On F0 trajectory for very high-quality speech manipulation2002

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.7th International Conference on Spoken Language Processing (ICSLP2002)

      Pages: 2397-2400

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Journal Article] On F0 trajectory for very high-quality speech manipulation2002

    • Author(s)
      Hideki Kawahara
    • Journal Title

      Proc.7th International Conference on Spoken Language Processing (ICSLP 2002)

      Pages: 2397-2400

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2004 Final Research Report Summary
  • [Book] Speech separation by human and machines2004

    • Author(s)
      Pierre Divenyi
    • Total Pages
      319
    • Publisher
      Kluwer Academic Pub.
    • Related Report
      2004 Annual Research Report
  • [Publications] 貫名真澄, 河原英紀: "発話時の頭部周辺での音声の伝達特性について"日本音響学会誌. Vol.59,No.5. 256-260 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Hisami Matsui, Hideki Kawahara: "Investigation of Emotionally Morphed Speech Perception and its Structure Using a High Quality Speech Manipulation System Prod"Eurospeech'03. 2113-2116 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Hideki Kawahara: "Exemplar-based Voice Quality Analysis and Control using a High Quality Auditory Morphing Procedure based on STRAIGHT"VOQUAL'03, ISCA Tutorial and Research Workshop. 109-114 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Hideki Kawahara, Hisami Matsui: "AUDITORY MORPHING BASED ON AN ELASTIC PERCEPTUAL DISTANCE METRIC IN AN INTERFERENCE-FREE TIME-FREQUENCY REPRESENTATION"ICASSP'2003. 6-10 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Ryuichirou Yanaga, Hideki Kawahara: "Logarithmic temporal axis manipulation and its application for measuring auditory contributions in FO control using a transformed auditory feedback procedure"J.Acoust.Soc.Am.. 114. 2458-2458 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Hideki Kawahara, Hideki Banno, Toshio Irino, Parham Zolfaghari: "ALGORITHM AMALGAM : MORPHING WAVEFORM BASED METHODS, SINUISOIDAL MODELS AND STRAIGHT"Proc.ICASSP'2004. (採録決定). (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] Hideki Kawahara: "Systematic Downgrading for Investigating "Naturaless" in Synthesized singing using STRAIGHT : A High Quality VOCODER"143th MEETING OF THE ACOUSTICAL SOCIETY OF AMERICA. Vol.111, No.5, Pt.2. 2334-2334 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] Hideki Kawahara, Parham Zolfaghari, Alain de Cheveigne: "ON F0 TRAJECTORY OPTIMIZATION FOR VERY HIGH-QUALITY SPEECH MANIPULATION"Proceedings of JCSLP 2002. Volume4. 2397-2400 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] Masumi Nukina, Hideki Kawahara: "Cross spectral measurement of head related speech transfer functions using speaker's own voice"The Journal of the Acoustical Society of America. Volume112, Issue5. 2324-2324 (2003)

    • Related Report
      2002 Annual Research Report
  • [Publications] Hisami Matsui, Hideki Kawahara: "Auditorily motivated elastic spectral distance and its application to emotional morphing of portrayal speech"The Journal of the Acoustical Society of America. Volume112, Issue5. 2323-2323 (2002)

    • Related Report
      2002 Annual Research Report
  • [Publications] Hideki Kawahara, Hisami Matsui: "Auditory morphing based on an elastic perceptual distance metric in an interference free time-frequency representation"Proceedings of ICASSP 2003. Vol.1. 256-259 (2003)

    • Related Report
      2002 Annual Research Report
  • [Publications] 貫名真澄, 河原英紀: "発話時の頭部周辺での音声の伝達特性について"日本音響学会誌. Vol.59, no.5. 256-260 (2003)

    • Related Report
      2002 Annual Research Report

URL: 

Published: 2002-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi