Machine learning based on sparsity-inducing regularization for matrices

• Project/Area Number: 22700138
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Single-year Grants
• Research Field: Intelligent informatics
• Research Institution: The University of Tokyo
• Principal Investigator: TOMIOKA Ryota 東京大学, 大学院・情報理工学系研究科, 助教 (70518282)
• Project Period (FY): 2010 – 2012
• Project Status: Completed (Fiscal Year 2012)
• Budget Amount: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000); Fiscal Year 2012: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000); Fiscal Year 2011: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2010: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
• Keywords: 機械学習 / スパース性 / 正則化 / 行列 / テンソル / スパース正則化 / スパース / 最適化

Research Abstract

The outcomes of this research project can be summarized as follows:1. I have extended the dual augmented Lagrangian (DAL) algorithm to deal with spectral regularization for matrices and proposed the M-DAL algorithm(ICML2010). The super-linear convergence of DAL and M-DAL algorithms wasproven and published in JMLR. A review of DAL and related algorithms has beenpublished as part of “Optimization for Machine Learning” (MIT Press). I have alsomade the code publicly available to promote its use in wider research communities. 2. In order to deal with non-numerical data, I have extended DAL to handle multiplekernel learning with thousands of kernels. This was published in MachineLearning Journal. 3. I have extended the framework to spectral regularization for higher-order tensors and analyzed its statistical performance. This was presented at NIPS2011.

Research Products

• [Journal Article] Global analytic solution of fully-observed variational Bayesian matrix factorization. (2013)
  • Author(s): S. Nakajima, M. Sugiyama, S. Babacan,and R. Tomioka.
  • Journal Title: J. Mach.Learn. Res. Volume: 14 Pages: 1-37
• [Journal Article] Global Analytic Solution of Fully-observed Variational Bayesian Matrix Factorization (2013)
  • Author(s): Shinichi Nakajima, Masashi Sugiyama, S. Derin Babacan, Ryota Tomioka
  • Journal Title: Journal of Machine Learning Research Volume: 14 Pages: 1-37
  • Peer Reviewed
• [Journal Article] Discovering emerging topics in social streams via link anomaly detection. (2012)
  • Author(s): T. Takahashi, R. Tomioka, and K. Yamanishi
  • Journal Title: IEEE Transactions on Knowledge and Data Engineering. Volume: (Accepted.)
• [Journal Article] Discovering Emerging Topics in Social Streams via Link Anomaly Detection (2012)
  • Author(s): Toshimitsu Takahashi, Ryota Tomioka, Kenji Yamanishi
  • Journal Title: IEEE Transactions on Knowledge and Data Engineering Volume: 未定 Issue: 1 Pages: 120-130
  • DOI: 10.1109/tkde.2012.239
  • Peer Reviewed
• [Journal Article] Super-linear convergence of dual augmented Lagrangian algorithm for sparse learning. (2011)
  • Author(s): R. Tomioka, T. Suzuki, and M. Sugiyama
  • Journal Title: Journal of Machine Learning Research Volume: 12 Pages: 1537-1586
• [Journal Article] SpicyMKL: A Fast Algorithm for Multiple Kernel Learning with Thousands of Kernels. (2011)
  • Author(s): T. Suzuki and R. Tomioka
  • Journal Title: Machine Learning Volume: 85 Pages: 77-108
• [Journal Article] SpicyMKL: A Fast Algorithm for Multiple Kernel Learning with Thousands of Kernels (2011)
  • Author(s): Taiji Suzuki and Ryota Tomioka
  • Journal Title: DOI 10.1007/s10994-011-5252-9 Volume: 85 Issue: 1-2 Pages: 77-108
  • DOI: 10.1007/s10994-011-5252-9
  • Peer Reviewed
• [Journal Article] Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparse Learning (2011)
  • Author(s): 冨岡亮太
  • Journal Title: Journal of Machine Learning Research Volume: (印刷中(掲載確定))
  • Peer Reviewed
• [Presentation] A Bayesian Analysis of the Radioactive Releases of Fukushima. (2012)
  • Author(s): R. Tomioka and M. Morup
  • Organizer: In JMLUR Workshop and Conference Proceedings 22 (AISTATS2012)
• [Presentation] A combinatorial algebraic approach for the identifiability of low-rank matrix completion. (2012)
  • Author(s): F. Kiraly and R. Tomioka
  • Organizer: In Proc. 29th International Conference on Machine Learning
• [Presentation] Perfect dimensionality recovery by variational Bayesian PCA. (2012)
  • Author(s): S. Nakajima, R. Tomioka, M. Sugiyama,and S. Babacan
  • Organizer: In Advances in Neural Information Processing Systems
• [Presentation] A Combinatorial Algebraic Approach for the Identifiability of Low-Rank Matrix Completion (2012)
  • Author(s): Franz Kiraly and Ryota Tomioka
  • Organizer: The 29th International Conference on Machine Learning (ICML2012)
  • Place of Presentation: エジンバラ・イギリス
• [Presentation] Perfect Dimensionality Recovery by Variational Bayesian PCA (2012)
  • Author(s): Shinichi Nakajima, Ryota Tomioka, Masashi Sugiyama, S. Derin Babacan
  • Organizer: Neural Information Processing Systems 2012 (NIPS2012)
  • Place of Presentation: レイクタホ・アメリカ
• [Presentation] A Bayesian Analysis of the Radioactive Releases of Fukushima (2012)
  • Author(s): Ryota Tomioka and Morten Mørup
  • Organizer: Artificial Intelligence and Statistics 2012 (AISTATS2012)
  • Place of Presentation: ラパルマ・スペイン
• [Presentation] Discovering emerging topics in social streams via link anomaly detection. (2011)
  • Author(s): T. Takahashi, R. Tomioka, and K.Yamanishi.
  • Organizer: In Proc. of the 11th International Conference on Data Mining(ICDM2011)
• [Presentation] Statistical Performance of Convex Tensor Decomposition. (2011)
  • Author(s): R Tomioka, T. Suzuki, K. Hayashi, and H. Kashima
  • Organizer: In Advances in Neural Information Processing Systems
• [Presentation] Statistical Performance of Convex Tensor Decomposition (2011)
  • Author(s): Ryota Tomioka, Taiji Suzuki, Kohei Hayashi, and Hisashi Kashima
  • Organizer: Neural Information Processing Systems 2011 (NIPS2011)
  • Place of Presentation: グラナダ・スペイン
• [Presentation] Discovering Emerging Topics in Social Streams via Link Anomaly Detection (2011)
  • Author(s): Toshimitsu Takahashi, Ryota Tomioka, and Kenji Yamanishi
  • Organizer: IEEE International Conference on Data Mining 2011 (ICDM2011)
  • Place of Presentation: バンクーバー・カナダ
• [Presentation] A Fast Augmented Lagrangian Algorithm for Learning Low-Rank Matrices (2010)
  • Author(s): 冨岡亮太
  • Organizer: 27th Annual International Conference on Machine Learning (ICML 2010)
  • Place of Presentation: イスラエル・ハイファ
  • Year and Date: 2010-06-22
• [Presentation] A fast augmented Lagrangian algorithm for learning low-rank matrices. (2010)
  • Author(s): R. Tomioka, T. Suzuki, M. Sugiyama, and H. Kashima.
  • Organizer: In Proc. the 27th Annual International Conference on Machine Learning (ICML2010)
  • Place of Presentation: Omnipress
• [Book] Augmented Lagrangian methods for learning, selecting, and combining features. (2011)
  • Author(s): R. Tomioka, T. Suzuki, and M. Sugiyama.
  • Publisher: MIT Press.
• [Book] Augmented Lagrangian Methods for Learning, Selecting, and Combining Features (S.Sra, S.Nowozin, S.J.Wright編集"Optimization for Machine Learning"の一部) (2011)
  • Author(s): 冨岡亮太
  • Publisher: MIT Press(印刷中(掲載確定))
• [Remarks]
  • URL: http://www.ibis.t.u-tokyo.ac.jp/ryotat/
• [Remarks] 双対拡張ラグランジュ法
  • URL: http://www.ibis.t.u-tokyo.ac.jp/ryotat/dal/
• [Remarks] DAL
  • URL: http://www.ibis.t.u-tokyo.ac.jp/ryotat/dal/
• [Remarks] Trace norm regularization
  • URL: http://www.ibis.t.u-tokyo.ac.jp/RyotaTomioka/Softwares/Tensor
• [Remarks] Ph.D. Course
  • URL: http://www.imm.dtu.dk/Forskning/ISP/Undervisning/02901_2012.aspx
• [Remarks] DAL
  • URL: http://www.ibis.t.u-tokyo.ac.jp/ryotat/dal/
• [Remarks] Trace norm regularization
  • URL: http://www.ibis.t.u-tokyo.ac.jp/RyotaTomioka/Softwares/Tensor
• [Remarks] Ph.D. Course
  • URL: http://www.imm.dtu.dk/Forskning/ISP/Undervisning/02901_2011.aspx
• [Remarks]
  • URL: http://www.ibis.t.u-tokyo.ac.jp/RyotaTomioka/Publications