• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of Riemannian conjugate gradient methods and their applications to large-scale problems

Research Project

Project/Area Number 16K17647
Research Category

Grant-in-Aid for Young Scientists (B)

Allocation TypeMulti-year Fund
Research Field Foundations of mathematics/Applied mathematics
Research InstitutionKyoto University (2017-2018)
Tokyo University of Science (2016)

Principal Investigator

SATO Hiroyuki  京都大学, 情報学研究科, 特定准教授 (80734433)

Research Collaborator AIHARA Kensuke  
KASAI Hiroyuki  
SATO Kazuhiro  
Project Period (FY) 2016-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥3,900,000 (Direct Cost: ¥3,000,000、Indirect Cost: ¥900,000)
Fiscal Year 2018: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2017: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2016: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Keywords最適化 / アルゴリズム / 数値線形代数 / 機械学習 / 制御工学 / 応用数学 / 幾何学 / 数理工学
Outline of Final Research Achievements

The generalized Stiefel manifold has important applications including the canonical correlation analysis. This research proposed a new retraction and its efficient implementation on this manifold, which is essential in Riemannian optimization methods such as the conjugate gradient method. As applications to control engineering, this research also proposed new algorithms for system identification and optimal model reduction problems, some of which are based on the Riemannian conjugate gradient method. Furthermore, this research generalized stochastic optimization methods, which are effective for large-scale optimization problems in machine learning with big data, to Riemannian manifolds together with their convergence analyses and numerical verification of their performances.

Academic Significance and Societal Importance of the Research Achievements

本研究の成果によって,機械学習や制御工学などにおいて現れる最適化問題の一部をより高速に解くことができるようになった.また,リーマン多様体上の共役勾配法や確率的最適化手法の一般論を発展させたことによって,それらが適用できるような大規模な問題をより効果的に解くための土台が築かれたといえる.個別の問題に対して,本研究で提案したアルゴリズムに基づくアプローチを構築することによって,諸分野における技術のさらなる発展が期待される.

Report

(4 results)
  • 2018 Annual Research Report   Final Research Report ( PDF )
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (57 results)

All 2019 2018 2017 2016 Other

All Int'l Joint Research (2 results) Journal Article (18 results) (of which Int'l Joint Research: 4 results,  Peer Reviewed: 13 results,  Open Access: 9 results,  Acknowledgement Compliant: 3 results) Presentation (37 results) (of which Int'l Joint Research: 10 results,  Invited: 4 results)

  • [Int'l Joint Research] Microsoft(インド)

    • Related Report
      2018 Annual Research Report
  • [Int'l Joint Research] Technical University of Kaiserslautern(ドイツ)

    • Related Report
      2018 Annual Research Report
  • [Journal Article] Riemannian stochastic variance reduced gradient algorithm2019

    • Author(s)
      Hiroyuki Sato, Hiroyuki Kasai, and Bamdev Mishra
    • Journal Title

      SIAM Journal on Optimization

      Volume: 印刷中 Issue: 2 Pages: 1444-1472

    • DOI

      10.1137/17m1116787

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Cholesky QR-based retraction on the generalized Stiefel manifold2019

    • Author(s)
      Hiroyuki Sato, Kensuke Aihara
    • Journal Title

      Computational Optimization and Applications

      Volume: 72 Issue: 2 Pages: 293-308

    • DOI

      10.1007/s10589-018-0046-7

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 幾何学的な最適化アルゴリズムとその応用2019

    • Author(s)
      佐藤寛之
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 2094 Pages: 55-64

    • Related Report
      2018 Annual Research Report
    • Open Access
  • [Journal Article] Riemannian Newton's Method on the Grassmann Manifold Exploiting the Quotient Structure2018

    • Author(s)
      佐藤 寛之,相原 研輔
    • Journal Title

      Transactions of the Japan Society for Industrial and Applied Mathematics

      Volume: 28 Issue: 4 Pages: 205-241

    • DOI

      10.11540/jsiamt.28.4_205

    • NAID

      130007552770

    • ISSN
      2424-0982
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Practically feasible recommender systems for cold start problems2018

    • Author(s)
      Mimu Kawai, Takayuki Shiohama, and Hiroyuki Sato
    • Journal Title

      Proceedings of 2018 5th Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE)

      Volume: 印刷中

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Riemannian stochastic recursive gradient algorithm2018

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Journal Title

      Proceedings of the 35th International Conference on Machine Learning (ICML 2018),Proceedings of Machine Learning Research

      Volume: 80 Pages: 2521-2529

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis2018

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Journal Title

      Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018), Proceedings of Machine Learning Research

      Volume: 84 Pages: 269-278

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] 独立低ランクテンソル分析:非負値性・低ランク性・独立性に基づくブラインド音源分離の統一理論2018

    • Author(s)
      吉井和佳,佐藤寛之,坂東宜昭,中村栄太,河原達也
    • Journal Title

      信学技報

      Volume: 118 Pages: 37-44

    • Related Report
      2018 Annual Research Report
  • [Journal Article] リーマン多様体上の最適化に基づく離散時間線形システム同定アルゴリズム2018

    • Author(s)
      佐藤寛之,佐藤一宏
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 2069 Pages: 153-165

    • Related Report
      2018 Annual Research Report
    • Open Access
  • [Journal Article] Structure preserving H^2 optimal model reduction based on Riemannian trust-region method2018

    • Author(s)
      Kazuhiro Sato and Hiroyuki Sato
    • Journal Title

      IEEE Transactions on Automatic Control

      Volume: 63 Issue: 2 Pages: 505-512

    • DOI

      10.1109/tac.2017.2723259

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Riemannian Newton-type methods for joint diagonalization on the Stiefel manifold with application to independent component analysis2017

    • Author(s)
      Hiroyuki Sato
    • Journal Title

      Optimization

      Volume: 66 Issue: 12 Pages: 2211-2231

    • DOI

      10.1080/02331934.2017.1359592

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Riemannian optimal system identification algorithm for linear MIMO systems2017

    • Author(s)
      Hiroyuki Sato and Kazuhiro Sato
    • Journal Title

      IEEE Control Systems Letters

      Volume: 1 Issue: 2 Pages: 376-381

    • DOI

      10.1109/lcsys.2017.2719163

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Supervised-topic-model-based hybrid filtering for recommender systems2017

    • Author(s)
      Mimu Kawai, Takayuki Shiohama, and Hiroyuki Sato
    • Journal Title

      Proceedings of the 2017 2nd Intl Conf on Big Data, Cloud Computing, and Data Science Engineering

      Volume: - Pages: 272-277

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] 一般化シュティーフェル多様体上のレトラクションとその効果的な実装について2017

    • Author(s)
      佐藤寛之,相原研輔
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 2027 Pages: 125-134

    • Related Report
      2017 Research-status Report
    • Open Access
  • [Journal Article] 直交制約つき最適化問題に対するリーマン多様体上の確率的分散縮小勾配法2017

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Journal Title

      京都大学数理解析研究所講究録

      Volume: 2027 Pages: 135-143

    • Related Report
      2017 Research-status Report
    • Open Access / Int'l Joint Research
  • [Journal Article] リーマン多様体上の最適化の理論と応用2017

    • Author(s)
      佐藤寛之
    • Journal Title

      応用数理

      Volume: 27 Pages: 21-30

    • NAID

      130007043034

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Journal Article] A matrix-free implementation of Riemannian Newton's method on the Stiefel manifold2016

    • Author(s)
      Kensuke Aihara and Hiroyuki Sato
    • Journal Title

      Optimization Letters

      Volume: 印刷中 Issue: 8 Pages: 1729-1741

    • DOI

      10.1007/s11590-016-1090-9

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] A new H^2 optimal model reduction method based on Riemannian conjugate gradient method2016

    • Author(s)
      Hiroyuki Sato and Kazuhiro Sato
    • Journal Title

      Proceedings of the 55th IEEE Conference on Decision and Control

      Volume: - Pages: 5762-5768

    • DOI

      10.1109/cdc.2016.7799155

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Presentation] リーマン多様体上の最適化におけるレトラクションの理論と応用2019

    • Author(s)
      佐藤寛之,相原研輔
    • Organizer
      日本応用数理学会2019年研究部会連合発表会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Practically feasible recommender systems for cold start problems2018

    • Author(s)
      Mimu Kawai, Takayuki Shiohama, and Hiroyuki Sato
    • Organizer
      5th Asia-Pacific World Congress on Computer Science and Engineering 2018 (APWC on CSE 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Riemannian stochastic recursive gradient algorithm2018

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Organizer
      The 35th International Conference on Machine Learning (ICML 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Stochastic recursive gradient on Riemannian manifolds2018

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Organizer
      Geometry in Machine Learning (GiMLi 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis2018

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Organizer
      The 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] リーマン多様体上の幾何学的な最適化手法とその応用2018

    • Author(s)
      佐藤寛之
    • Organizer
      Interaction between Pure and Applied Mathematics 2018
    • Related Report
      2018 Annual Research Report
    • Invited
  • [Presentation] 独立低ランクテンソル分析:非負値性・低ランク性・独立性に基づくブラインド音源分離の統一理論2018

    • Author(s)
      吉井和佳,佐藤寛之,坂東宜昭,中村栄太,河原達也
    • Organizer
      第21回情報論的学習理論ワークショップ (IBIS2018)
    • Related Report
      2018 Annual Research Report
  • [Presentation] AdaGrad のリーマン多様体上への拡張について2018

    • Author(s)
      高柳俊,佐藤寛之,山下信雄
    • Organizer
      日本オペレーションズ・リサーチ学会 関西支部 SSOR2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] 対称性を備えた連続時間線形システム同定に対するリーマン多様体上の最適化によるアプローチ2018

    • Author(s)
      佐藤寛之,佐藤一宏,Tobias Damm
    • Organizer
      日本オペレーションズ・リサーチ学会2018年秋季研究発表会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 幾何学的最適化に基づく対称性をもつ連続時間線形システムの新しい同定法2018

    • Author(s)
      佐藤寛之,佐藤一宏,Tobias Damm
    • Organizer
      日本応用数理学会2018年度年会
    • Related Report
      2018 Annual Research Report
  • [Presentation] リーマン多様体上の確率的最適化の発展2018

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Organizer
      RIMS共同研究(公開型)「高度情報化社会に向けた数理最適化の新潮流」
    • Related Report
      2018 Annual Research Report
    • Invited
  • [Presentation] グラスマン多様体上の商構造に基づくニュートン方程式とその解法2018

    • Author(s)
      佐藤寛之,相原研輔
    • Organizer
      第47回数値解析シンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] Solving a Newton equation on the Stiefel manifold with matrix-free Krylov subspace methods2018

    • Author(s)
      Kensuke Aihara and Hiroyuki Sato
    • Organizer
      2018 SIAM Conference on Parallel Processing for Scientific Computing
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] グラスマン多様体上のニュートン方程式の水平空間での表現とその求解について2018

    • Author(s)
      佐藤寛之,相原研輔
    • Organizer
      日本応用数理学会2018年研究部会連合発表会
    • Related Report
      2017 Research-status Report
  • [Presentation] 一般化シュティーフェル多様体上の最適化におけるレトラクションについて2017

    • Author(s)
      佐藤寛之,相原研輔
    • Organizer
      日本オペレーションズ・リサーチ学会 2017年春季研究発表会
    • Place of Presentation
      沖縄県市町村自治会館(沖縄県那覇市)
    • Year and Date
      2017-03-15
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian optimal system identification algorithm for linear MIMO systems2017

    • Author(s)
      Hiroyuki Sato and Kazuhiro Sato
    • Organizer
      The 56th IEEE Conference on Decision and Control
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Supervised-topic-model-based hybrid filtering for recommender systems2017

    • Author(s)
      Mimu Kawai, Takayuki Shiohama, and Hiroyuki Sato
    • Organizer
      2nd International Conference on Big Data, Cloud Computing, and Data Science Engineering
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] 幾何学的な最適化アルゴリズムとその応用2017

    • Author(s)
      佐藤寛之
    • Organizer
      RIMS共同研究(公開型)「数値解析学の最前線-理論・方法・応用-」
    • Related Report
      2017 Research-status Report
    • Invited
  • [Presentation] リーマン多様体上の最適化を用いた離散時間線形システム同定2017

    • Author(s)
      佐藤寛之,佐藤一宏
    • Organizer
      日本オペレーションズ・リサーチ学会2017年秋季研究発表会
    • Related Report
      2017 Research-status Report
  • [Presentation] 幾何学的最適化に基づく離散時間線形システム同定アルゴリズム2017

    • Author(s)
      佐藤寛之,佐藤一宏
    • Organizer
      日本応用数理学会2017年度年会
    • Related Report
      2017 Research-status Report
  • [Presentation] リーマン多様体上の最適化に基づく離散時間線形システム同定アルゴリズム2017

    • Author(s)
      佐藤寛之,佐藤一宏
    • Organizer
      RIMS共同研究(公開型)「数理最適化の発展:モデル化とアルゴリズム」
    • Related Report
      2017 Research-status Report
  • [Presentation] 幾何学的な最適化理論とその周辺2017

    • Author(s)
      佐藤寛之
    • Organizer
      芝浦工業大学システム理工学部数理科学科 第63回談話会
    • Related Report
      2017 Research-status Report
    • Invited
  • [Presentation] A new H^2 optimal model reduction method based on Riemannian conjugate gradient method2016

    • Author(s)
      Hiroyuki Sato and Kazuhiro Sato
    • Organizer
      The 55th IEEE Conference on Decision and Control
    • Place of Presentation
      Las Vegas (USA)
    • Year and Date
      2016-12-12
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Riemannian stochastic variance reduced gradient on Grassmann manifold2016

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Organizer
      The 9th NIPS Workshop on Optimization for Machine Learning
    • Place of Presentation
      Barcelona (Spain)
    • Year and Date
      2016-12-10
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] QR分解に基づくシュティーフェル多様体上のレトラクションの一般化について2016

    • Author(s)
      佐藤寛之,相原研輔
    • Organizer
      日本応用数理学会「行列・固有値問題の解法とその応用」研究部会第22回研究会
    • Place of Presentation
      東京大学(東京都文京区)
    • Year and Date
      2016-11-25
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian stochastic variance reduced gradient (R-SVRG)2016

    • Author(s)
      笠井裕之,佐藤寛之,Bamdev Mishra
    • Organizer
      第19回情報論的学習理論ワークショップ (IBIS2016)
    • Place of Presentation
      京都大学(京都府京都市)
    • Year and Date
      2016-11-16
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian stochastic variance reduced gradient for large-scale data applications2016

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Organizer
      2016年電子情報通信学会ソサイエティ大会
    • Place of Presentation
      北海道大学(北海道札幌市)
    • Year and Date
      2016-09-20
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian stochastic variance reduced gradient and its convergence analysis2016

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Organizer
      日本オペレーションズ・リサーチ学会 2016年秋季研究発表会
    • Place of Presentation
      山形大学(山形県山形市)
    • Year and Date
      2016-09-14
    • Related Report
      2016 Research-status Report
  • [Presentation] SVRG 法のグラスマン多様体上への拡張とその行列補完問題への応用2016

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Organizer
      日本応用数理学会2016年度年会
    • Place of Presentation
      北九州国際会議場(福岡県北九州市)
    • Year and Date
      2016-09-12
    • Related Report
      2016 Research-status Report
  • [Presentation] コレスキーQR分解に基づく一般化シュティーフェル多様体上のレトラクション2016

    • Author(s)
      相原研輔,佐藤寛之
    • Organizer
      日本応用数理学会2016年度年会
    • Place of Presentation
      北九州国際会議場(福岡県北九州市)
    • Year and Date
      2016-09-12
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian stochastic variance reduced gradient on Grassmann manifold2016

    • Author(s)
      笠井裕之,佐藤寛之,Bamdev Mishra
    • Organizer
      第15回情報科学技術フォーラム
    • Place of Presentation
      富山大学(富山県富山市)
    • Year and Date
      2016-09-07
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian stochastic variance reduced gradient and its applications to machine learning2016

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Organizer
      2016年度統計関連学会連合大会
    • Place of Presentation
      金沢大学(石川県金沢市)
    • Year and Date
      2016-09-04
    • Related Report
      2016 Research-status Report
  • [Presentation] 直交制約つき最適化問題に対するリーマン多様体上の確率的分散縮小勾配法2016

    • Author(s)
      佐藤寛之,笠井裕之,Bamdev Mishra
    • Organizer
      京都大学数理解析研究所研究集会「最適化技法の最先端と今後の展開」
    • Place of Presentation
      京都大学数理解析研究所(京都府京都市)
    • Year and Date
      2016-08-25
    • Related Report
      2016 Research-status Report
  • [Presentation] 一般化シュティーフェル多様体上のレトラクションとその効果的な実装について2016

    • Author(s)
      佐藤寛之,相原研輔
    • Organizer
      京都大学数理解析研究所研究集会「最適化技法の最先端と今後の展開」
    • Place of Presentation
      京都大学数理解析研究所(京都府京都市)
    • Year and Date
      2016-08-25
    • Related Report
      2016 Research-status Report
  • [Presentation] Riemannian stochastic variance reduced gradient on Grassmann manifold2016

    • Author(s)
      Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra
    • Organizer
      The Fifth International Conference on Continuous Optimization
    • Place of Presentation
      政策研究大学院大学(東京都港区)
    • Year and Date
      2016-08-06
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Riemannian stochastic variance reduced gradient for large-scale machine learning2016

    • Author(s)
      笠井裕之,佐藤寛之,Bamdev Mishra
    • Organizer
      電子情報通信学会 モバイルネットワークとアプリケーション研究会 (MoNA)
    • Place of Presentation
      神戸大学(兵庫県神戸市)
    • Year and Date
      2016-08-01
    • Related Report
      2016 Research-status Report
  • [Presentation] 非線形共役勾配法のリーマン多様体上への拡張とその応用について2016

    • Author(s)
      佐藤寛之
    • Organizer
      応用解析研究会 ~ 可積分系から計算数学まで ~
    • Place of Presentation
      天満研修センター(大阪府大阪市)
    • Year and Date
      2016-05-19
    • Related Report
      2016 Research-status Report

URL: 

Published: 2016-04-21   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi