Singing information processing considering diversity of singing voice utilizing singing big data

2018 Fiscal Year Final Research Report

• Project/Area Number: 17K12721
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Multi-year Fund
• Research Field: Perceptual information processing
• Research Institution: National Institute of Advanced Industrial Science and Technology
• Principal Investigator: Nakano Tomoyasu 国立研究開発法人産業技術総合研究所, 情報・人間工学領域, 主任研究員 (10572927)
• Project Period (FY): 2017-04-01 – 2019-03-31
• Keywords: 歌声情報処理 / 信号処理 / 機械学習 / インタフェース / 情報可視化

Outline of Final Research Achievements

Fundamental technologies to model the diversity of singing voices using a large-scale data set (i.e., singing big data) were developed. Specifically, to deal with singing voice in music, methods based on probabilistic models and deep learning were developed to improve the performance of vocal activity detection, lyric synchronization, F0 (pitch) estimation, and voice separation. A fundamental technology to estimate the spectral envelope of unaccompanied singing voice with high accuracy was also developed. In order to apply those results, we realized an interface to visualize "what and how to sing" at the same time, and a new singing voice visualization interface for annotation using repetition of singing voice. Furthermore, in order to apply these methods, we implemented an interface that simultaneously visualizes "what was uttered and how the words were expressed" and a new singing voice visualization interface for annotation that utilizes repetition of singing voice.

Free Research Field

歌声情報処理

Academic Significance and Societal Importance of the Research Achievements

音楽に含まれる歌声は処理が難しく未解決で本質的な課題が多い。一方で、産業・文化の両面で主要なコンテンツである音楽における最も重要な要素の一つである。したがって、学術的および産業応用的な観点からの注目度が高い。本研究の成果における歌詞同期、音高推定、歌声分離等の混合音中の歌声分析技術は、世界的に活発に研究されており、その性能向上は学術的・産業応用的に意義がある。また、そのような要素技術の性能向上が、社会・エンドユーザの音楽活動を豊かにするためには、適切なインタフェースや可視化が必要不可欠であり、その新しい技術を実現した点でも社会的に意義がある。