| Research Abstract |
The objective of this research was to newly explore and establish techniques for signal and information processing of polyphonic music signals and objects with polyphonic structure and/or parallel temporal structure via approaches based on probabilistic models, aiming to deal mainly with music signals and information containing polyphony (e.g. chords) or simultaneity (e.g. accompaniment). Typical applications include automatic music transcription, automatic arrangement, music information retrieval, and music modification. Concerning multipitch analysis based on the harmonic-temporal-structured model, we established the HTC (Harmonic-Temporal Clustering) method, which estimates multiple pitches by estimating the parameters of the acoustic object model comprised of a mixture of Gaussian distributions on the time-frequency plane with harmonic structure and temporal continuity. Based on this method, we developed a technique for analyzing polyphonic signals and converting them into MIDI dat
… More
a. As for rhythm and tempo estimation, we developed an automatic transcription technique which reconstructs the underlying score from MIDI information through recognition of rhythm and estimation of tempo as a latent variable, based on a HMM (Hidden Markov Model). We also developed an automatic accompaniment system which plays an accompaniment following the user playing one part of polyphony with changing tempo, making mistakes, and jumping to arbitrary points in the music. Concerning separation of multiple signals with distinct periods in the time domain, we developed a method for solving a single-channel source separation problem which aims for separation of signals with distinct fundamental periods from their mixture through the auxiliary function method, an extension of the EM algorithm. We also started research on multipitch analysis based on nonnegative matrix factorization, and developed a technique for estimating timbre vectors and note activity intervals by solving the minimization problem of an error between observation and decomposition in such a way that the result is as sparse as possible, within the framework of factorization of the observed spectrogram matrix into a product of a matrix containing as few basis vectors as possible and a note activity matrix. As for computational harmony theory, we made attempt to arrange the harmony theory taught in music schools, a basic compositional theory, so that computers can handle it, based on HMM and stochastic context free grammar. This laid the groundwork for automatic harmonic analysis, automatic harmonization of melodies, automatic composition based on harmonics, etc. This research is characterized as applying the methodology of speech recognition to the music information processing area, as well as applying the developed methods to speech recognition and hand-written character recognition. Less
|
