• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Mathematics and Practical Algorithms for machine Learning methods with non-convex losses

Research Project

Project/Area Number 16K00044
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Statistical science
Research InstitutionTokyo Institute of Technology (2017-2019)
Nagoya University (2016)

Principal Investigator

Kanamori Takafumi  東京工業大学, 情報理工学院, 教授 (60334546)

Project Period (FY) 2016-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2018: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2017: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2016: ¥1,950,000 (Direct Cost: ¥1,500,000、Indirect Cost: ¥450,000)
Keywords機械学習 / 数理統計学 / 最適化 / 変数選択 / ロバスト統計 / ロバスト
Outline of Final Research Achievements

The purpose is to promote research from a unified viewpoint of optimization of non-convex loss functions for problems of statistical learning such as robust estimation and sparse modeling. We constructed a practically efficient learning algorithm and built the mathematical foundation of non-convex learning. In particular, for robust support vector machines with a non-convex loss function, we mathematically analyzed the properties of the local optimal solutions. Useful mathematical concepts such as breakdown point have been applied to analyze the properties of learning algorithms with non-convex functions for classification tasks. As a result, it was clarified that even a local optimal solution theoretically achieves high prediction accuracy even under strong contamination Furthermore, we proposed a statistical analysis method for discrete data, and showed that the non-convex optimization can be efficiently calculated by replacing it with local calculation.

Academic Significance and Societal Importance of the Research Achievements

非凸損失を用いる統計的学習は一般に,統計的性質は優れているが,最適化のための計算が非常に困難であることが知られている.最適化の困難を解決することができれば,,データに大きなノイズが含まれる場合でも,非常にロバストな統計的推論を高い計算効率で実行することが可能になる.本研究では,判別分析や離散確率分布の推定という重要な問題クラスに対して,非凸損失による効率的な学習アルゴリズムを提案し,その数理的性質を詳しく研究した.その結果,理論,実装の両面から提案法の有効性を確認することができた.

Report

(5 results)
  • 2019 Annual Research Report   Final Research Report ( PDF )
  • 2018 Research-status Report
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (39 results)

All 2019 2018 2017 2016 Other

All Int'l Joint Research (5 results) Journal Article (14 results) (of which Int'l Joint Research: 3 results,  Peer Reviewed: 14 results,  Open Access: 7 results) Presentation (16 results) (of which Int'l Joint Research: 6 results,  Invited: 8 results) Book (4 results)

  • [Int'l Joint Research] University of Bristol(英国)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] Max Plank Institute(ドイツ)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] University College London/University of Bristol(英国)

    • Related Report
      2018 Research-status Report
  • [Int'l Joint Research] Max Planck Institutes(ドイツ)

    • Related Report
      2018 Research-status Report
  • [Int'l Joint Research] Harvard University(米国)

    • Related Report
      2018 Research-status Report
  • [Journal Article] Spectral Embedded Deep Clustering2019

    • Author(s)
      Wada Yuichiro、Miyamoto Shugo、Nakagama Takumi、Andeol Leo、Kumagai Wataru、Kanamori Takafumi
    • Journal Title

      Entropy

      Volume: 21 Issue: 8 Pages: 795-795

    • DOI

      10.3390/e21080795

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Robust Label Prediction via Label Propagation and Geodesic k-Nearest Neighbor in Online Semi-Supervised Learning.2019

    • Author(s)
      Y. Wada, S. Su, W. Kumagai, T. Kanamori,
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E102-D Pages: 1537-1545

    • NAID

      130007686445

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Variable Selection for Nonparametric Learning with Power Series Kernels2019

    • Author(s)
      Matsui Kota、Kumagai Wataru、Kanamori Kenta、Nishikimi Mitsuaki、Kanamori Takafumi
    • Journal Title

      Neural Computation

      Volume: 31 Issue: 8 Pages: 1718-1750

    • DOI

      10.1162/neco_a_01212

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Model Description of Similarity-Based Recommendation Systems2019

    • Author(s)
      Kanamori Takafumi、Osugi Naoya
    • Journal Title

      Entropy

      Volume: 21 Issue: 7 Pages: 702-702

    • DOI

      10.3390/e21070702

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Numerical Study of Reciprocal Recommendation with Domain Matching2019

    • Author(s)
      K. Sudo, N. Osugi, T. Kanamori
    • Journal Title

      Japanese Journal of Statistics and Data Science

      Volume: 2 Issue: 1 Pages: 221-240

    • DOI

      10.1007/s42081-019-00033-3

    • NAID

      210000170705

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Risk bound of transfer learning using parametric feature mapping and its application to sparse coding2019

    • Author(s)
      Kumagai Wataru、Kanamori Takafumi
    • Journal Title

      Machine Learning

      Volume: 108 Issue: 11 Pages: 1975-2008

    • DOI

      10.1007/s10994-019-05805-2

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Mode-Seeking Clustering and Density Ridge Estimation via Direct Estimation of Density-Derivative-Ratios2018

    • Author(s)
      H. Sasaki, T. Kanamori, A. Hyvarinen, and Masashi Sugiyama,
    • Journal Title

      Journal of Machine Learning Research

      Volume: 18

    • Related Report
      2018 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Graph-based Composite Local Bregman Divergences on Discrete Sample Spaces2017

    • Author(s)
      T. Kanamori, T. Takenouchi
    • Journal Title

      Neural Networks

      Volume: 95 Pages: 44-56

    • DOI

      10.1016/j.neunet.2017.06.005

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Robustness of learning algorithms using hinge loss with outlier indicators2017

    • Author(s)
      Kanamori Takafumi、Fujiwara Shuhei、Takeda Akiko
    • Journal Title

      Neural Networks

      Volume: 94 Pages: 173-191

    • DOI

      10.1016/j.neunet.2017.07.005

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Parallel Distributed Block Coordinate Descent Methods based on Pairwise Comparison Oracle2017

    • Author(s)
      K. Matsui, W. Kumagai, T. Kanamori
    • Journal Title

      Journal of Global Optimization

      Volume: 69 Issue: 1 Pages: 1-21

    • DOI

      10.1007/s10898-016-0465-x

    • NAID

      110009971451

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Statistical Inference with Unnormalized Discrete Models and Localized Homogeneous Divergences2017

    • Author(s)
      T. Takenouchi, T. Kanamori
    • Journal Title

      Journal of Machine Learning Research

      Volume: 18 Pages: 1-26

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] DC Algorithm for Extended Robust Support Vector Machine2017

    • Author(s)
      Fujiwara Shuhei、Takeda Akiko、Kanamori Takafumi
    • Journal Title

      Neural Computation

      Volume: 29 Issue: 5 Pages: 1406-1438

    • DOI

      10.1162/neco_a_00958

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Breakdown point of robust support vector machines2017

    • Author(s)
      Kanamori, T., Fujiwara, S., Takeda, A.
    • Journal Title

      Entropy

      Volume: 19 Issue: 2 Pages: 83-83

    • DOI

      10.3390/e19020083

    • NAID

      110009971425

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Efficiency bound of local Z-estimators on discrete sample spaces2016

    • Author(s)
      Kanamori, T.
    • Journal Title

      Entropy

      Volume: 18 Issue: 7 Pages: 273-273

    • DOI

      10.3390/e18070273

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] Fisher Efficient Inference of Intractable Models.2019

    • Author(s)
      S. Liu, T. Kanamori, W. Jitkrittum, Y. Chen,
    • Organizer
      The Neural Information Processing Systems (NeurIPS 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Foundations of transfer learning and its application to multi-center prognostic prediction.2019

    • Author(s)
      K. Matsui, W. Kumagai, K. Kanamori, M. Nisikimi, S. Matsui, T. Kanamori,
    • Organizer
      WNAR/IMS/JR Annual Meeting, Portland, Oregon, USA
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Statistical inference with unnormalized models2018

    • Author(s)
      Takafumi Kanamori
    • Organizer
      International Symposium on Statistical Theory and Methodology for Large Complex Data
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] 変数選択付きカーネル密度比推定に基づく多施設の予後予測解析2018

    • Author(s)
      松井 孝太,熊谷 亘,金森 研太,錦見 満暁,金森 敬文
    • Organizer
      統計関連学会連合学会
    • Related Report
      2018 Research-status Report
  • [Presentation] Variable Selection Consistency in Kernel Methods using shrinkage parameters2018

    • Author(s)
      Takafumi Kanamori, Kota Matsui, Wataru Kumagai, Kenta Kanamori
    • Organizer
      統計関連学会連合学会
    • Related Report
      2018 Research-status Report
  • [Presentation] カーネル法における変数選択2018

    • Author(s)
      金森敬文
    • Organizer
      大規模統計モデリングと計算統計V
    • Related Report
      2018 Research-status Report
    • Invited
  • [Presentation] カーネル法における変数選択の一致性2018

    • Author(s)
      金森敬文
    • Organizer
      研究集会「実験計画法ならびに情報数理と関連する組合せ構造 2018」
    • Related Report
      2018 Research-status Report
    • Invited
  • [Presentation] Parallel Distributed Block Coordinate Descent Methods Based on Pairwise Comparison Oracle2017

    • Author(s)
      K. Matsui, W. Kumagai, T. Kanamori
    • Organizer
      the 2017 INFORMS ANNUAL MEETING
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] Estimating Density Ridges by Direct Estimation of Density-Derivative-Ratios2017

    • Author(s)
      Hiroaki Sasaki, Takafumi Kanamori and Masashi Sugiyama,
    • Organizer
      the 20th International Conference on Artificial Intelligence and Statistics
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] パラメータ転移学習におけるリスク上界2017

    • Author(s)
      熊谷 亘,金森 敬文
    • Organizer
      統計関連学会連合大会
    • Related Report
      2017 Research-status Report
  • [Presentation] 局所情報による統計的推論2017

    • Author(s)
      金森 敬文
    • Organizer
      統計関連学会連合大会
    • Related Report
      2017 Research-status Report
  • [Presentation] Bregman divergence and its Applications2016

    • Author(s)
      金森敬文
    • Organizer
      情報理論とその応用シンポジウム
    • Place of Presentation
      岐阜
    • Year and Date
      2016-12-13
    • Related Report
      2016 Research-status Report
    • Invited
  • [Presentation] Statistical Inference using Graph-based Divergences on Discrete Sample Spaces2016

    • Author(s)
      Takafumi Kanamori
    • Organizer
      International Symposium on Statistical Analysis for Large Complex Data
    • Place of Presentation
      筑波大学
    • Year and Date
      2016-11-21
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] グラフ上の局所ブレグマンダイバージェンスによる統計的推定2016

    • Author(s)
      金森敬文, 竹之内高志
    • Organizer
      情報論的学習理論研究会
    • Place of Presentation
      京都大学
    • Year and Date
      2016-11-16
    • Related Report
      2016 Research-status Report
  • [Presentation] 離散空間上のグラフ構造に基つく 局所ブレグマンダイバージェンス2016

    • Author(s)
      金森敬文, 竹之内高志
    • Organizer
      統計関連学会連合大会
    • Place of Presentation
      金沢大学
    • Year and Date
      2016-09-05
    • Related Report
      2016 Research-status Report
  • [Presentation] ダイバージェンスによる統計的推論2016

    • Author(s)
      金森敬文, 藤澤洋徳
    • Organizer
      統計関連学会連合大会
    • Place of Presentation
      金沢大学
    • Year and Date
      2016-09-05
    • Related Report
      2016 Research-status Report
    • Invited
  • [Book] Pythonで学ぶ統計的機械学習2018

    • Author(s)
      金森敬文
    • Total Pages
      252
    • Publisher
      オーム社
    • ISBN
      9784274223051
    • Related Report
      2018 Research-status Report
  • [Book] Rによる機械学習入門2017

    • Author(s)
      金森 敬文
    • Total Pages
      260
    • Publisher
      オーム社
    • ISBN
      9784274221125
    • Related Report
      2017 Research-status Report
  • [Book] 機械学習のための連続最適化2016

    • Author(s)
      金森 敬文, 鈴木 大慈, 竹内 一郎, 佐藤 一誠
    • Total Pages
      213
    • Publisher
      講談社
    • Related Report
      2016 Research-status Report
  • [Book] モデリングの諸相2016

    • Author(s)
      室田一雄, 池上敦子, 土谷隆, 山下浩, 蒲地 政文, 畔上秀幸, 斉藤努, 枇々木規雄, 滝根哲哉, 金森敬文
    • Total Pages
      35
    • Publisher
      近代科学社
    • Related Report
      2016 Research-status Report

URL: 

Published: 2016-04-21   Modified: 2021-02-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi