• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Signal processing technology based on deep learning and application to singing voice and musical instrument sound generation

Research Project

Project/Area Number 18K11163
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 60010:Theory of informatics-related
Research InstitutionNagoya Institute of Technology

Principal Investigator

Oura Keiichiro  名古屋工業大学, 工学(系)研究科(研究院), 研究員 (20588579)

Project Period (FY) 2018-04-01 – 2021-03-31
Project Status Completed (Fiscal Year 2020)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2020: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2018: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Keywords信号処理 / ディープラーニング / 歌声合成 / 音声合成 / 楽器音生成 / 楽器音合成
Outline of Final Research Achievements

For singing voices and instrument sounds, we proceeded research on acoustic modeling about automatic selection method of training data, modeling method of speech waveform itself, and end-to-end structure capable of direct conversion from musical score to waveform, etc. and publish some of them as academic papers. Among them, the waveform generation from periodic / aperiodic signals based on deep learning by applying the cycle structure of CycleGAN which show high performance in the field of image conversion has achieved results such as receiving the KIYOSHI AWAYA academic encouragement award from the acoustical society of Japan and the Microsoft informatics research award from the information processing society of Japan.

Academic Significance and Societal Importance of the Research Achievements

現状のほとんどの音声関連技術には,従来型のデジタル信号処理理論を基礎としており,従来型のデジタル信号処理理論は音声関連の研究分野では最も根本的な考え方として広く普及しているが,このような変換・処理で取り扱える枠組みの中に制限されていたため,モデル構造に関する過度の制約による性能限界があった.本研究は,このような状況にブレークスルーをもたらすため,近年急速に技術革新が進んでいる深層学習に基づいた音声波形の直接モデル化手法を新たに開拓しようとするものである.

Report

(4 results)
  • 2020 Annual Research Report   Final Research Report ( PDF )
  • 2019 Research-status Report
  • 2018 Research-status Report
  • Research Products

    (21 results)

All 2021 2020 2019 2018

All Presentation (21 results) (of which Int'l Joint Research: 9 results,  Invited: 2 results)

  • [Presentation] 周期・非周期成分の分離に基づくニューラルボコーダによる音声波形のモデル化の検討2021

    • Author(s)
      法野行哉, 高木信二, 橋本佳, 大浦圭一郎, 南角吉彦, 徳田恵一
    • Organizer
      日本音響学会2021年春季研究発表会, pp. 861-864, 日本, 2021年3月.
    • Related Report
      2020 Annual Research Report
  • [Presentation] DNNに基づく音声ボコーダにおける周期・非周期成分のモデル化の検討2020

    • Author(s)
      法野行哉, 高木信二, 橋本佳, 大浦圭一郎, 南角吉彦, 徳田恵一
    • Organizer
      日本音響学会2020年秋季研究発表会, pp. 759-760, 日本, 2020年9月.
    • Related Report
      2020 Annual Research Report
  • [Presentation] 楽譜時間情報を用いたアテンション機構に基づく歌声合成の検討2019

    • Author(s)
      村田舜馬, 藤本崇人, 法野行哉, 高木信二, 橋本佳, 大浦圭一郎, 南角吉彦, 徳田恵一
    • Organizer
      日本音響学会2019年秋季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] Singing voice synthesis based on generative adversarial networks2019

    • Author(s)
      Yukiya Hono, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, Keiichi Tokuda
    • Organizer
      2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Speaker-dependent WaveNet-based delay-free ADPCM speech coding2019

    • Author(s)
      Takenori Yoshimura, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, Keiichi Tokuda
    • Organizer
      2019 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] 周期・非周期信号を用いたDNNに基づくリアルタイム音声ボコーダ2019

    • Author(s)
      大浦圭一郎, 中村和寛, 橋本佳, 南角吉彦, 徳田恵一
    • Organizer
      情報処理学会研究報告
    • Related Report
      2019 Research-status Report
  • [Presentation] 周期・非周期信号を用いた敵対的生成ネットワークに基づくリアルタイム音声ボコーダ2019

    • Author(s)
      大浦圭一郎, 高木信二, 中村和寛, 橋本佳, 南角吉彦, 徳田恵一
    • Organizer
      日本音響学会2019年秋季研究発表会
    • Related Report
      2019 Research-status Report
  • [Presentation] Deep neural network based real-time speech vocoder with periodic and aperiodic inputs2019

    • Author(s)
      Keiichiro Oura, Kazuhiro Nakamura, Kei Hashimoto, Yoshihiko Nankaku, Keiichi Tokuda
    • Organizer
      10th ISCA Speech Synthesis Workshop (SSW10)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] 歌声合成におけるニューラルボコーダの比較検討2019

    • Author(s)
      和田蒼汰, 法野行哉, 高木信二, 橋本佳, 大浦圭一郎, 南角吉彦, 徳田恵一
    • Organizer
      音声研究会
    • Related Report
      2019 Research-status Report
  • [Presentation] 統計的歌声合成技術とその実用化2019

    • Author(s)
      大浦圭一郎
    • Organizer
      日本AI音楽学会
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] 統計的パラメトリック音声合成技術とその実用化2019

    • Author(s)
      大浦圭一郎
    • Organizer
      情報処理学会音学シンポジウム
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] Speaker-dependent WaveNet-based delay-free adpcm speech coding2019

    • Author(s)
      Takenori Yoshimura, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, and Keiichi Tokuda
    • Organizer
      ICASSP 2019
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Singing voice synthesis based on generative adversarial networks2019

    • Author(s)
      Yukiya Hono, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, and Keiichi Tokuda
    • Organizer
      ICASSP 2019
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] 周期・非周期信号から駆動するディープニューラルネットに基づく音声ボコーダ2019

    • Author(s)
      大浦圭一郎,中村和寛,橋本佳,南角吉彦,徳田恵一
    • Organizer
      日本音響学会春季研究発表会
    • Related Report
      2018 Research-status Report
  • [Presentation] 敵対的ネットワークを用いた歌声合成の検討2019

    • Author(s)
      法野行哉,橋本佳,大浦圭一郎,南角吉彦,徳田恵一
    • Organizer
      日本音響学会春季研究発表会
    • Related Report
      2018 Research-status Report
  • [Presentation] Singing voice conversion using posted waveform data on music social media2018

    • Author(s)
      Koki Senda, Yukiya Hono, Kei Sawada, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, and Keiichi Tokuda
    • Organizer
      APSIPA 2018
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Recent development of the DNN-based singing voice synthesis system - sinsy2018

    • Author(s)
      Yukiya Hono, Shumma Murata, Kazuhiro Nakamura, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, and Keiichi Tokuda
    • Organizer
      APSIPA 2018
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Speech synthesis using WaveNet vocoder based on periodic/aperiodic decomposition2018

    • Author(s)
      Takahto Fujimoto, Takenori Yoshimura, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, and Keiichi Tokuda
    • Organizer
      APSIPA 2018
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] WaveNet-based zero-delay lossless speech coding2018

    • Author(s)
      Takenori Yoshimura, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, and Keiichi Tokuda
    • Organizer
      SLT 2018
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Deep neural networkに基づく歌声合成システム - Sinsy2018

    • Author(s)
      法野行哉,村田舜馬,中村和寛,橋本佳,大浦圭一郎,南角吉彦,徳田恵一
    • Organizer
      日本音響学会秋季研究発表会
    • Related Report
      2018 Research-status Report
  • [Presentation] 周期・非周期成分の分離に基づくWaveNetボコーダを用いた音声合成2018

    • Author(s)
      藤本崇人,吉村建慶,橋本佳,大浦圭一郎,南角吉彦,徳田恵一
    • Organizer
      日本音響学会秋季研究発表会
    • Related Report
      2018 Research-status Report

URL: 

Published: 2018-04-23   Modified: 2022-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi