Theory and applications of cross-data-type machine learning methods

• Project/Area Number: 22700289
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Single-year Grants
• Research Field: Statistical science
• Research Institution: The University of Tokyo
• Principal Investigator: SUZUKI Taiji 東京大学, 大学院・情報理工学系研究科, 助教 (60551372)
• Project Period (FY): 2010 – 2012
• Project Status: Completed (Fiscal Year 2012)
• Budget Amount: ¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000); Fiscal Year 2012: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2011: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2010: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
• Keywords: 統計的学習理論 / multiple kernel learning / スパース加法モデル / オンライン最適化 / 双対平均化法 / 近接勾配法 / スパース学習 / 正則化 / 機械学習 / 統計数学

Research Abstract

We have investigated statistical convergence properties of Multiple Kernel Learning (MKL) with various types of regularizations. Moreover, we proposed a Bayesian variant of MKL and showed that it has optimality without strong assumptions on the design which are assumed in conventional theoretical analysis of MKL. We also proposed an online optimization method that is useful for structured regularization. It was shown that the proposed algorithm converges in the mini-max optimal rate.

Research Products

• [Journal Article] Fast learning rate ofmultiple kernel learning:trade-off between sparsity and smoothness (2013)
  • Author(s): Taiji Suzuki, and Masashi
  • Journal Title: The Annals of Statistics
• [Journal Article] Computational complexity of kernel-based density-ratio estimation (2013)
  • Author(s): Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: A condition number analysis. Machine Learning Volume: vol.90 Pages: 431-460
• [Journal Article] Sufficient dimension reduction via squared-loss mutual information estimation (2013)
  • Author(s): Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: Neural Computation Volume: vol. 25 Pages: 725-758
  • NAID: 130008079589
• [Journal Article] Relative Density-Ratio Estimation for Robust Distribution Comparison (2013)
  • Author(s): Makoto Yamada, Taiji Suzuki, Takafumi Kanamori, Hirotaka Hachiya, and Masashi Sugiyama
  • Journal Title: Neural Computation Volume: 25 Issue: 5 Pages: 1324-1370
  • DOI: 10.1162/neco_a_00442
  • NAID: 10031099905
  • Peer Reviewed
• [Journal Article] Computational complexity of kernel-based density-ratio estimation: A condition number analysis (2013)
  • Author(s): Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: Machine Learning Volume: 90 Issue: 3 Pages: 431-460
  • DOI: 10.1007/s10994-012-5323-6
  • Peer Reviewed
• [Journal Article] Statistical analysis of kernel-based least-squares density-ratio estimation (2012)
  • Author(s): Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: Machine Learning Volume: 86 Issue: 3 Pages: 335-367
  • DOI: 10.1007/s10994-011-5266-3
  • Peer Reviewed
• [Journal Article] f-divergence estimation and two-sample homogeneity test under semiparametric density-ratio models (2012)
  • Author(s): Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: IEEE Transactions on Information Theory Volume: 58 Issue: 2 Pages: 708-720
  • DOI: 10.1109/tit.2011.2163380
  • Peer Reviewed
• [Journal Article] A Fast Algorithm for Multiple Kernel Learning with Thousands of Kernels (2011)
  • Author(s): Taiji Suzuki and Ryota Tomioka: SpicyMKL
  • Journal Title: Machine Learning Volume: vol. 85, issue 1 Pages: 77-108
• [Journal Article] Density ratio matching under the Bregman divergence: A unified framework of density ratio estimation (2011)
  • Author(s): Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori
  • Journal Title: Annals of the Institute of Statistical Mathematics Volume: 11 Issue: 5 Pages: 1-36
  • DOI: 10.1007/s10463-011-0343-8
  • NAID: 40019382740
  • Peer Reviewed
• [Journal Article] SpicyMKL: A Fast Algorithm for Multiple Kernel Learning with Thousands of Kernels (2011)
  • Author(s): Taiji Suzuki and Ryota Tomioka
  • Journal Title: DOI 10.1007/s10994-011-5252-9 Volume: 85 Issue: 1-2 Pages: 77-108
  • DOI: 10.1007/s10994-011-5252-9
  • Peer Reviewed
• [Journal Article] Least-Squares Two-Sample Test (2011)
  • Author(s): Masashi Sugiyama, Taiji Suzuki, Yuta Itoh, and Takafumi Kanamori
  • Journal Title: Neural Networks Volume: 24 Issue: 7 Pages: 735-751
  • DOI: 10.1016/j.neunet.2011.04.003
  • Peer Reviewed
• [Journal Article] Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparse Learning (2011)
  • Author(s): Ryota Tomioka, Taiji Suzuki, and Masashi Sugiyama
  • Journal Title: Journal of Machine Learning Research Volume: 12 Pages: 1537-1586
  • Peer Reviewed
• [Journal Article] Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search (2011)
  • Author(s): Masashi Sugiyama, Makoto Yamada, Paul von Bunau, Taiji Suzuki, Takafumi Kanamori, Motoaki Kawanabe
  • Journal Title: Neural Networks Volume: 24 Pages: 183-198
  • Peer Reviewed
• [Journal Article] Least-squares Independent Component Analysis (2011)
  • Author(s): Taiji Suzuki, Masashi Sugiyama
  • Journal Title: Neural Computation Volume: 23 Pages: 284-301
  • Peer Reviewed
• [Journal Article] On prior selection and covariate shift ofbeta-Bayesian prediction under alpha-divergence risk. (2010)
  • Author(s): Taiji Suzuki, and Fumiyasu Komaki
  • Journal Title: Communications in Statistics --- Theory and Methods Volume: 39(8) Pages: 1655-1673
• [Journal Article] Piecewise Affine Systems Modelling for Optimizing Hormonal Therapy of Prostate Cancer (2010)
  • Author(s): Taiji Suzuki, Nicholas Bruchovsky, Kazuyuki Aihara
  • Journal Title: Philosophical Transactions A of the Royal Society Volume: 368 Pages: 5045-5059
  • Peer Reviewed
• [Journal Article] Theoretical analysis of density ratio estimation (2010)
  • Author(s): Takafumi Kanamori, Taiji Suzuki, Masashi Sugiyama
  • Journal Title: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Volume: E93-A Pages: 787-798
  • NAID: 10026863929
  • Peer Reviewed
• [Journal Article] On prior selection and covariate shift of $￥beta$-Bayesian prediction under $￥alpha$-divergence risk (2010)
  • Author(s): Taiji Suzuki, Fumiyasu Komaki
  • Journal Title: Communications in Statistics---Theory and Methods Volume: 39 Pages: 1655-1673
  • Peer Reviewed
• [Journal Article] Relative Density-Ratio Estimation for Robust Distribution Comparison
  • Author(s): Makoto Yamada,Taiji Suzuki,Takafumi Kanamori, Hirotaka Hachiya, Masashi Sugiyama
  • Journal Title: Neural Computation
  • NAID: 10031099905
• [Presentation] Some convergence results on multiple kernel additive models.N (2012)
  • Author(s): Taiji Suzuki
  • Organizer: onparametric and High-dimensionalStatistics
  • Place of Presentation: Luminy, France
  • Year and Date: 2012-12-18
• [Presentation] 統計的学習理論チュートリア ル (2012)
  • Author(s): 鈴木大慈
  • Organizer: 第15回情報論的学 習理論ワークショップ (IBIS2012)
  • Place of Presentation: 筑波大学東京キャンパス文京校舎
  • Year and Date: 2012-11-07
• [Presentation] PAC-Bayesian Bound for Gaussian Process Regression and Multiple Kernel Additive Model (2012)
  • Author(s): Taiji Suzuki
  • Organizer: Conference on Learning Theory (COLT2012)
  • Place of Presentation: Edinburgh, Scotland
  • Year and Date: 2012-06-25
• [Presentation] Fast Learning Rate of Multiple Kernel Learning (2012)
  • Author(s): Taiji Suzuki and Masashi Sugiyama
  • Organizer: Trade-off between Sparsity and Smoothness. Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS2012)
  • Place of Presentation: La Palma, Canary Islands
  • Year and Date: 2012-04-21
• [Presentation] Density-Difference Estimation (2012)
  • Author(s): Masashi Sugiyama, Takafumi Kanamori, Taiji Suzuki, Marthinus Christoffel du Plessis, Song Liu, and Ichiro Takeuchi
  • Organizer: Advances in Neural Information Processing Systems (NIPS2012)
  • Place of Presentation: Lake Tahoe, Nevada, United States
• [Presentation] Some convergence results on multiple kernel additive models (2012)
  • Author(s): Taiji Suzuki
  • Organizer: Nonparametric and High-dimensional Statistics
  • Place of Presentation: Luminy, France
  • Invited
• [Presentation] A Conjugate Property between Loss Functions and Uncertainty Sets in Classification Problems (2012)
  • Author(s): Takafumi Kanamori, Akiko Takeda and Taiji Suzuki
  • Organizer: Conference on Learning Theory (COLT2012)
  • Place of Presentation: Edinburgh, Scotland
• [Presentation] Fast Learning Rate of Multiple Kernel Learning: Trade-off between Sparsity and Smoothness (2012)
  • Author(s): Taiji Suzuki and Masashi Sugiyama
  • Organizer: Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS2012)
  • Place of Presentation: La Palma, Canary Islands
• [Presentation] 統計的学習理論チュートリアル：基礎から応用まで (2012)
  • Author(s): 鈴木 大慈
  • Organizer: 第15回情報論的学習理論ワークショップ (IBIS2012)
  • Place of Presentation: 筑波大学東京キャンパス文京校舎
  • Invited
• [Presentation] ガウシアンプロセス回帰を用いた加法モデル推定のPAC-Bayesバウンド (2012)
  • Author(s): 鈴木 大慈
  • Organizer: 統計関連学会連合大会
  • Place of Presentation: 北海道大学
• [Presentation] ２値判別における損失関数と不確実性集合の共役性 (2012)
  • Author(s): 金森 敬文，武田 朗子 ，鈴木 大慈
  • Organizer: 統計関連学会連合大会
  • Place of Presentation: 北海道大学
• [Presentation] Unifying Framework for Fast Learning Rate of Non-Sparse Multiple Kernel Learning (2011)
  • Author(s): Taiji Suzuki
  • Organizer: Advances in Neural Information Processing Systems 24 (NIPS2011)
  • Place of Presentation: Granada, Spain.
  • Year and Date: 2011-12-12
• [Presentation] Fast Convergence Rate of Multiple Kernel Learning with Elastic-net Regularization (2011)
  • Author(s): 鈴木大慈, 冨岡亮太, 杉山将
  • Organizer: 第4回IBISML研究会
  • Place of Presentation: 阪大中之島センター
  • Year and Date: 2011-03-29
• [Presentation] カーネル密度比推定の統計的解析 (2011)
  • Author(s): 金森敬文, 鈴木大慈, 杉山将
  • Organizer: 第4回IBISML研究会
  • Place of Presentation: 阪大中之島センター
  • Year and Date: 2011-03-29
• [Presentation] Unifying Framework for Fast Learning Rate of Non-Sparse Multiple Kernel Learning (2011)
  • Author(s): Taiji Suzuki
  • Organizer: Advances in Neural Information Processing Systems 24 (NIPS2011)
  • Place of Presentation: Granada, Spain
• [Presentation] Statistical Performance of Convex Tensor Decomposition (2011)
  • Author(s): Ryota Tomioka, Taiji Suzuki, Kohei Hayashi and Hisashi Kashima
  • Organizer: Advances in Neural Information Processing Systems 24 (NIPS2011)
  • Place of Presentation: Granada, Spain
• [Presentation] Relative Density-Ratio Estimation for Robust Distribution Comparison (2011)
  • Author(s): Makoto Yamada, Taiji Suzuki, Takafumi Kanamori, Hirotaka Hachiya and Masashi Sugiyama
  • Organizer: Advances in Neural Information Processing Systems 24 (NIPS2011)
  • Place of Presentation: Granada, Spain
• [Presentation] 疎でないマルチプルカーネル学習の速い収束レートおよび最適な正則化に関して (2011)
  • Author(s): 鈴木 大慈
  • Organizer: 第14回情報論的学習理論ワークショップ (IBIS2011)
  • Place of Presentation: 奈良女子大学
• [Presentation] Multiple Kernel Learningの効率的計算法と統計的性質について (2011)
  • Author(s): 鈴木 大慈
  • Organizer: 応用数理学会若手の会単独研究会
  • Place of Presentation: 早稲田大学
• [Presentation] スパース正則化学習の学習性能,特にスパース性と汎化誤差の関係について (2010)
  • Author(s): 鈴木大慈
  • Organizer: 第13回情報論的学習理論ワークショップ(IBIS2010)
  • Place of Presentation: 東京大学
  • Year and Date: 2010-11-04
• [Presentation] A computational method to predict PSA evolution for androgen deprivation therapy (2010)
  • Author(s): Taiji Suzuki, Nicholas Bruchovsky, Kazuyuki Aihara
  • Organizer: Sixth International Symposium on Hormonal Oncogenesis
  • Place of Presentation: Sheraton Grande Tokyo Bay Hotel, Tokyo
  • Year and Date: 2010-09-14
• [Presentation] Elasticnet型正則化を用いたMultiple Kernel Learningの漸近的性質について (2010)
  • Author(s): 鈴木大慈, 冨岡亮太, 杉山将
  • Organizer: 統計関連学会連合大会
  • Place of Presentation: 早稲田大学
  • Year and Date: 2010-09-06
• [Presentation] 密度比の推定による2標本検定 (2010)
  • Author(s): 金森敬文, 鈴木大慈, 杉山将
  • Organizer: 統計関連学会連合大会
  • Place of Presentation: 早稲田大学
  • Year and Date: 2010-09-06
• [Presentation] An Efficient and General Augmented Lagrangian Algorithm for Learning Low-Rank Matrices (2010)
  • Author(s): Ryota Tomioka, Taiji Suzuki, Masashi Sugiyama, Hisashi Kashima
  • Organizer: 27th International Conference on Machine Learning International (ICML2010)
  • Place of Presentation: Haifa, Israel
  • Year and Date: 2010-06-22
• [Presentation] Conditional density estimation via least-squares density ratio estimation (2010)
  • Author(s): Masashi Sugiyama, Ichiro Takeuchi, Takafumi Kanamori, Taiji Suzuki, Hirotaka Hachiya, Daisuke Okanohara
  • Organizer: Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS2010)
  • Place of Presentation: Chia Laguna Resort, Sardinia, Italy
  • Year and Date: 2010-05-14
• [Presentation] Sufficient dimension reduction via squared-loss mutual information estimation (2010)
  • Author(s): Taiji Suzuki, Masashi Sugiyama
  • Organizer: Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS2010)
  • Place of Presentation: Chia Laguna Resort, Sardinia, Italy
  • Year and Date: 2010-05-13
• [Presentation] Direct density ratio estimation with dimensionality reduction (2010)
  • Author(s): Masashi Sugiyama, Satoshi Hara, Paul von Bunau, Taiji Suzuki, Takafumi Kanamori, Motoaki Kawanabe
  • Organizer: 2010 SIAM International Conference on Data Mining (SDM2010)
  • Place of Presentation: Columbus, Ohio, USA
  • Year and Date: 2010-04-30
• [Book] Cambridge UniversityPress (2012)
  • Author(s): Masashi Sugiyama, T aiji Suzuki, andTakafumi Kanamori
  • Total Pages: 329
  • Publisher: Density Ratio Estimationin Machine Learning
• [Book] Density Ratio Estimation in Machine Learning (2012)
  • Author(s): Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori
  • Total Pages: 329
  • Publisher: Cambridge University Press
• [Remarks]
  • URL: http://www.simplex.t.u-tokyo.ac.jp/~s-taiji/
• [Remarks] http://www.simplex.t.u-tokyo.ac.jp/~s-taiji/
• [Remarks] http://www.simplex.t.u-tokyo.ac.jp/~s-taiji/
• [Remarks]
  • URL: http://www.simplex.t.u-tokyo.ac.jp/~s-taiji/