• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

2010 Fiscal Year Final Research Report

Analysis, Recognition, Manipulation and Generation of Music Signal and Information based on Mathematical Models

Research Project

  • PDF
Project/Area Number 20240017
Research Category

Grant-in-Aid for Scientific Research (A)

Allocation TypeSingle-year Grants
Section一般
Research Field Perception information processing/Intelligent robotics
Research InstitutionThe University of Tokyo

Principal Investigator

SAGAYAMA Shigeki  The University of Tokyo, 大学院・情報理工学系研究科, 教授 (00303321)

Co-Investigator(Kenkyū-buntansha) ONO Nobutaka  東京大学, 大学院・情報理工学系研究科, 講師 (80334259)
NISHIMOTO Takuya  東京大学, 大学院・情報理工学系研究科, 助教 (80283696)
Co-Investigator(Renkei-kenkyūsha) KANEKO Hitomi  桐朋学園大学, 音楽学部, 准教授 (00408949)
Project Period (FY) 2008 – 2010
Keywords音楽信号解析 / 信号加工 / 音楽情報処理 / 多重音解析 / 調波時間クラスタリング / 非負値行列分解 / 調波音打楽器音信号分離 / 位相復元 / 速度変換 / ピッチ変換 / パート分離 / 歌声抽出 / 歌声消去 / 和声推定 / リズム解析 / 自動小節分割 / 楽曲ジャンル認識 / リズム構造推定 / Dynamic Bayesian Net音楽モデル / 自動採譜 / 作曲家推定 / 楽譜自動演奏 / 自動対位法 / 自動歌唱作曲 / 機能和声データベース
Research Abstract

An diverse research area has been actively investigated in analysis, recognition, manipulation and generation of music signal and information resulting in numerous excellent results. They include quite a wide range of research achievements such as multipitch estimation method (called HTC) based on a model of multiple concurrent music sounds and another method based on non-negative matrix factorization (NMF), harmonic-percussive sound separation (HPSS), high-quality signal manipulation (e.g., music tempo and pitch modification) based on restoration of phase components from power spectrogram, part separation form stereophonic music recordings, vocal sound extraction and deletion from ensemble music signals based on spectral fluctuation of human voice, chord estimation from music signals, automatic extraction of constituent rhythms (RhythmMap) in music signals and bar-line segmentation, music genre and mood recognition based on the previous method, rhythm estimation in polyphonic music signals and application to automatic music transcription, music modeling and automatic transcription based on dynamic Bayesian network to integrate hierarchical layers from music theory to signal observations, composer identification from score data, automatic music performance rendition using conditional random field, automatic counterpoint with imitation of music motives, automatic song composition based on intrinsic intonation of Japanese lyrics, and building functional harmony database. These achievements have been made public through journal papers, international conferences, review papers, domestic academic presentations, web service for automatic music composition, and public media.

  • Research Products

    (33 results)

All 2011 2010 2009 2008 Other

All Journal Article (7 results) Presentation (19 results) Book (2 results) Remarks (3 results) Patent(Industrial Property Rights) (2 results)

  • [Journal Article] Computational Auditory Induction as a Missing-Data Model-Fitting Problem with Bregman Divergence2011

    • Author(s)
      Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, Alain de Cheveign, Shigeki Sagayama
    • Journal Title

      Speech Communication Vol.53, Issue 5

      Pages: 658-676

  • [Journal Article] Beyond Timbral Statistics : Improving Music Classification Using Percussive Patterns and Bass Lines2011

    • Author(s)
      E.Tsunoo, G.Tzanetakis, N.Ono, S.Sagayama
    • Journal Title

      IEEE Transactions on Audio, Speech, and Language Processing 19, 4

      Pages: 1003-1014

  • [Journal Article] Speech Spectrum Modeling for Joint Estimation of Spectral Envelope and Fundamental Frequency2010

    • Author(s)
      Hirokazu Kameoka, Nobutaka Ono, Shigeki Sagayama
    • Journal Title

      IEEE Transactions on Audio Speech and Language Processing Vol.18, No.6

      Pages: 1507-1516

  • [Journal Article] 多重音解析と自動採譜2009

    • Author(s)
      亀岡弘和,嵯峨山茂樹
    • Journal Title

      情報処理 Vol.50, No.8

      Pages: 711-716

  • [Journal Article] 音楽音響信号の実時間加工技術2009

    • Author(s)
      小野順貴
    • Journal Title

      情報処理 Vol.50, No.8

      Pages: 717-722

  • [Journal Article] 和声解析・リズム認識・自動伴奏・運指決定2009

    • Author(s)
      嵯峨山茂樹
    • Journal Title

      情報処理 Vol.50, No.8

      Pages: 735-745

  • [Journal Article] 自動採譜技術の展望2008

    • Author(s)
      嵯峨山茂樹,亀岡弘和
    • Journal Title

      日本音響学会誌 Vol.64, No.12

      Pages: 715-720

  • [Presentation] Infinite-State Spectrum Model for Music Signal Analysis2011

    • Author(s)
      Masahiro Nakano, Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.ICASSP 2011 International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2011)
    • Place of Presentation
      (accepted for publication)
    • Year and Date
      20110500
  • [Presentation] Nonnegative Matrix Factorization with Markov-chained Bases for Modeling Time-varying patterns in Music Spectrograms2010

    • Author(s)
      Masahiro Nakano, Jonathan Le Roux, Hirokazu Kameoka, Yu Kitano, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of LVA/ICA, pp.149-156
    • Year and Date
      20100900
  • [Presentation] Convergence-guaranteed multiplicative algorithms for nonnegative matrix factorization with beta-divergence2010

    • Author(s)
      Masahiro Nakano, Hirokazu Kameoka, Jonathan Le Roux, Yu Kitano, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.MLSP, pp.283-288
    • Year and Date
      20100800
  • [Presentation] Performance rendering for polyphonic piano music with a combination of probabilistic models for melody and harmony2010

    • Author(s)
      Tae Hun Kim, Satoru Fukayama, Takuya Nishimoto, Shigeki Sagayama
    • Organizer
      Proceedings of SMC, pp 23-30
    • Year and Date
      20100700
  • [Presentation] Melody Line Estimation in Homophonic Music Audio Signals Based on Temporal-Variability of Melodic Source2010

    • Author(s)
      Hideyuki Tachibana, Takuma Ono, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of ICASSP, AE-P7.7, pp.425-428
    • Year and Date
      20100300
  • [Presentation] Music Mood Classification by Rhythm and Bass-line Unit Pattern Analysis2010

    • Author(s)
      Emiru Tsunoo, Taichi Akase, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of ICASSP, AE-P4.5, pp.265-268
    • Year and Date
      20100300
  • [Presentation] HMM-Based Approach for Automatic Chord Detection Using Refined Acoustic Features2010

    • Author(s)
      Yushi Ueda, Yuuki Uchiyama, Takuya Nishimoto, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of ICASSP, SS-L5.4, pp.5518-5521
    • Year and Date
      20100300
  • [Presentation] 非負値行列分解における時間包絡の単旋律性と基底の類似性に基づく音楽音響信号の楽器音分離2010

    • Author(s)
      村尾一真,中野允裕,北野佑,小野順貴,嵯峨山茂樹
    • Organizer
      日本音響学会春季研究発表会講演集,3-8-3,pp.1001-1002
    • Year and Date
      20100300
  • [Presentation] Note detection with dynamic Bayesian networks as a postanalysis step for NMF-based multiple pitch estimation techniques2009

    • Author(s)
      Stanislaw Andrzej Raczynski, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.WASPAA, pp.49-52
    • Year and Date
      20091000
  • [Presentation] Orpheus : Automatic Composition System Considering Prosody of Japanese Lyrics2009

    • Author(s)
      Satoru Fukayama, Kei Nakatsuma, Shinji Sako, Yu-ichiro Yonebayashi, Tae Hun Kim, Qin Si Wei, Takuho Nakano, Takuya Nishimoto, Shigeki Sagayama
    • Organizer
      Entertainment Computing-ICEC 2009, pp.309-310
    • Year and Date
      20090900
  • [Presentation] Extending Nonnegative Matrix Factorization-a discussion in the context of mulipitch frequency estimation of musical signals2009

    • Author(s)
      Stanislaw Andrzej Raczynski, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of EUSIPCO, pp.934-938
    • Year and Date
      20090800
  • [Presentation] Spectrogram consistency and its application to phase reconstruction2009

    • Author(s)
      Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      2009-MUS-81, No.8
    • Year and Date
      20090700
  • [Presentation] Rhythm Map : Extraction of Unit Rhythmic Patterns and Analysis of Rhythmic Structure from Music Acoustic Signals2009

    • Author(s)
      Emiru Tsunoo, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of ICASSP, pp.185-188
    • Year and Date
      20090400
  • [Presentation] What Can Speech Researchers Bring to Music Processing?2008

    • Author(s)
      Shigeki Sagayama
    • Organizer
      Proc.of 2008 International Symposium on Chinese Spoken Language Processing, pp.19-20
    • Place of Presentation
      (invited plenary talk)
    • Year and Date
      20081200
  • [Presentation] Toward automatic music transcription from audio input2008

    • Author(s)
      Shigeki Sagayama, Hirokazu Kameoka, Haruto Takeda
    • Organizer
      Journal of Acoustical Society of America, p.26
    • Place of Presentation
      (invited)
    • Year and Date
      20081100
  • [Presentation] A Real-time Equalizer of Harmonic and Percussive Components in Music Signals2008

    • Author(s)
      Nobutaka Ono, Ken-ichi Miyamoto, Hirokazu Kameoka, Shigeki Sagayama
    • Organizer
      Proc.of ISMIR, pp.139-144
    • Year and Date
      20080900
  • [Presentation] Orpheus:歌詞の韻律に基づいた自動作曲システム2008

    • Author(s)
      深山覚,中妻啓,米林裕一郎,酒向慎司,西本卓也,小野順貴,嵯峨山茂樹
    • Organizer
      情報処理学会研究報告,2008-MUS-76,No.30,pp.179-184
    • Year and Date
      20080800
  • [Presentation] Harmonic-Temporal-Timbral Clustering (HTTC) for the Analysis of Multi-instrument Polyphonic Music Signals2008

    • Author(s)
      Ken-ichi Miyamoto, Hirokazu Kameoka, Takuya Nishimoto, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of ICASSP, pp.113-116
    • Year and Date
      20080400
  • [Presentation] Auxiliary Function Approach to Parameter Estimation of Constrained Sinusoidal Model2008

    • Author(s)
      Hirokazu Kameoka, Nobutaka Ono, Shigeki Sagayama
    • Organizer
      Proc.of 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2008), pp.29-32
    • Year and Date
      20080400
  • [Book] Advances in Music Information Retrieval, ser. Studies in Computational Intelligence(Harmonic and Percussive Sound Separation and its Application to MIR-related Tasks Z.W.Ras and A.Wieczorkowska, Eds.)2010

    • Author(s)
      Nobutaka Ono, Kenichi Miyamoto, Hirokazu Kameoka, Jonathan Le Roux, Yuuki Uchiyama, Emiru Tsunoo, Takuya Nishimoto, Shigeki Sagayama
    • Publisher
      Springer
  • [Book] Handbook of Computer Systems for Expressive Music Performance(Statistical Approach to Automatic Expressive Rendition of Polyphonic Piano Music Alexis Kirke and Eduardo R.Miranda, Eds.)

    • Author(s)
      Tae Hun Kim, Satoru Fukayama, Takuya Nishimoto, Shigeki Sagayama
    • Publisher
      Springer-Verlag(to appear)
  • [Remarks] ホームページ等

    • URL

      http://hil.t.u-tokyo.ac.jp/

  • [Remarks] 研究室HP

    • URL

      http://ngs.hil.t.u-tokyo.ac.jp/~orpheus/cgi-bin/

  • [Remarks] 自動作曲サービス

  • [Patent(Industrial Property Rights)] 音声を含む信号の処理方法及び装置2009

    • Inventor(s)
      小野順貴、嵯峨山茂樹、橘秀幸
    • Industrial Property Rights Holder
      東京大学
    • Industrial Property Number
      特許出願, 特願2009-054766
    • Filing Date
      2009-03-09
  • [Patent(Industrial Property Rights)] 音信号の分離方法2008

    • Inventor(s)
      嵯峨山茂樹、小野順貴、亀岡弘和、宮本賢一、ルルージョナトン
    • Industrial Property Rights Holder
      東京大学
    • Industrial Property Number
      特許出願, PCT/JP2008/065287、特願2008-054826
    • Filing Date
      2008-08-27

URL: 

Published: 2012-01-26   Modified: 2016-01-20  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi