• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Developing efficient algorithms for nonconvex non smooth optimization and its application to machine learning

Research Project

Project/Area Number 19H04069
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 60020:Mathematical informatics-related
Research InstitutionThe University of Tokyo

Principal Investigator

Takeda Akiko  東京大学, 大学院情報理工学系研究科, 教授 (80361799)

Co-Investigator(Kenkyū-buntansha) ロウレンソ ブルノ・フィゲラ  統計数理研究所, 数理・推論研究系, 准教授 (80778720)
Liu Tianxiang  国立研究開発法人理化学研究所, 革新知能統合研究センター, 特別研究員 (90835216)
Metel MichaelRos  国立研究開発法人理化学研究所, 革新知能統合研究センター, 特別研究員 (40839081)
Project Period (FY) 2019-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥14,820,000 (Direct Cost: ¥11,400,000、Indirect Cost: ¥3,420,000)
Fiscal Year 2022: ¥2,990,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥690,000)
Fiscal Year 2021: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
Fiscal Year 2020: ¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2019: ¥4,940,000 (Direct Cost: ¥3,800,000、Indirect Cost: ¥1,140,000)
Keywords非凸非平滑最適化 / 機械学習 / DC最適化 / 最悪反復計算量 / 大域収束性 / 確率最適化 / 錐最適化問題 / ランダム行列理論 / 2段階最適化問題 / ホモトピー法 / ペナルティ法 / 確率的DCアルゴリズム / 低ランクスパース最適化
Outline of Research at the Start

L_0ノルムやランク関数は非凸関数であり,そのような関数を目的関数や制約式に含むスパース最適化問題は,一般にはNP困難であることが知られている.最近,我々は,スパース最適化問題を含む広いクラスの問題に対して,勾配計算に基づく効率的な解法SDCAを提案した.非凸スパース最適化法の研究は始まったばかりで,実用化に向けて様々な課題が残されている.本研究では,SDCAの効率化,そして,最適解により近い解を求めるためのSDCAの改良を目指す.効率性と高い近似精度は相反する特徴であり,両方を兼ね揃えた解法の開発は簡単ではないものの,これまでの大域最適化における研究成果を生かして解法の開発を行なう.

Outline of Final Research Achievements

The sparse optimization problem is a typical example of a nonsmooth nonconvex optimization problem, which has two intractable characteristics: nonconvexity and non-differentiability. In this research project, we have developed a solution method that combines the Successive Difference-of-Convex Approximation method (SDCA) with the homotopy method to achieve both "efficiency" and "high approximation accuracy". Furthermore, by simultaneously changing the parameters and performing iterative update to find a stationary point to improve computational efficiency, we were able to construct an algorithm with a theoretical convergence rate that is comparable to that of ordinary nonlinear optimization methods.

Academic Significance and Societal Importance of the Research Achievements

非凸非平滑最適化問題に対して, 効率性,高い近似精度の両方を兼ね揃えた解法の提案することを目標に掲げて研究を遂行した.解くべき問題を凸計画問題から元問題の非凸最適化問題へと変形しつつ最適解を求めていくという,ホモトピー法のアイディアを取り入れることで,両方の特徴を兼ね備えた解法の構築が可能になった.現実問題には非凸最適化問題として定式化される場合がしばしばあるが,既存の非凸最適化法を用いると,用いる初期解によって得られる解がかなり異なるため,応用上使いにくい場合も多い.本研究成果により,この非凸性による欠点がある程度解消されるため,十分に社会的意義もある成果と自己評価している.

Report

(5 results)
  • 2023 Final Research Report ( PDF )
  • 2022 Annual Research Report
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • 2019 Annual Research Report
  • Research Products

    (47 results)

All 2024 2023 2021 2020 2019 Other

All Int'l Joint Research (8 results) Journal Article (25 results) (of which Int'l Joint Research: 7 results,  Peer Reviewed: 24 results,  Open Access: 9 results) Presentation (11 results) (of which Int'l Joint Research: 6 results,  Invited: 5 results) Remarks (3 results)

  • [Int'l Joint Research] The Hong Kong Polytechnic University(中国)

    • Related Report
      2022 Annual Research Report
  • [Int'l Joint Research] Vrije Universiteit Brussel(ベルギー)

    • Related Report
      2020 Annual Research Report
  • [Int'l Joint Research] The Hong Kong Polytechnic University(中国)

    • Related Report
      2020 Annual Research Report
  • [Int'l Joint Research] 香港理工大学(中国)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] サウザンプトン大学(英国)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] シンガポール国立大学(シンガポール)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] 梨花女子大学(韓国)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] ブリュッセル自由大学(ベルギー)

    • Related Report
      2019 Annual Research Report
  • [Journal Article] Stable Linear System Identification with Prior Knowledge by Riemannian Sequential Quadratic Optimization2024

    • Author(s)
      Mitsuaki Obara , Kazuhiro Sato , Hiroki Sakamoto , Takayuki Okuno , Akiko Takeda
    • Journal Title

      IEEE Transactions on Automatic Control

      Volume: 69 Issue: 3 Pages: 2060-2066

    • DOI

      10.1109/tac.2023.3318195

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Random projections of linear and semidefinite problems with linear inequalities2023

    • Author(s)
      Poirion Pierre-Louis、Lourenco Bruno F.、Takeda Akiko
    • Journal Title

      Linear Algebra and its Applications

      Volume: 664 Pages: 24-60

    • DOI

      10.1016/j.laa.2023.01.013

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A study on modularity density maximization: Column generation acceleration and computational complexity analysis2023

    • Author(s)
      Issey Sukeda, Atsushi Miyauchi, and Akiko Takeda
    • Journal Title

      European Journal of Operational Research

      Volume: - Issue: 2 Pages: 516-528

    • DOI

      10.1016/j.ejor.2023.01.061

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Complexity analysis of interior-point methods for second-order stationary points of nonlinear semidefinite optimization problems2023

    • Author(s)
      Shun Arahata , Takayuki Okuno , Akiko Takeda
    • Journal Title

      Computational Optimization and Applications

      Volume: 86 Issue: 2 Pages: 555-598

    • DOI

      10.1007/s10589-023-00501-3

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Doubly majorized algorithm for sparsity-inducing optimization problems with regularizer-compatible constraints2023

    • Author(s)
      Liu Tianxiang、Pong Ting Kei、Takeda Akiko
    • Journal Title

      Computational Optimization and Applications

      Volume: 86 Issue: 2 Pages: 521-553

    • DOI

      10.1007/s10589-023-00503-1

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Robust Gaussian Process Regression with the Trimmed Marginal Likelihood2023

    • Author(s)
      Daniel Andrade, Akiko Takeda
    • Journal Title

      Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence

      Volume: PMLR 216 Pages: 67-76

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Primal-dual subgradient method for constrained convex optimization problems2021

    • Author(s)
      Michael R. Metel, Akiko Takeda
    • Journal Title

      Optimization Letters

      Volume: 15 Issue: 4 Pages: 1491-1504

    • DOI

      10.1007/s11590-021-01728-x

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Stochastic Proximal Methods for Non-Smooth Non-Convex Constrained Sparse Optimization2021

    • Author(s)
      Metel Michael R.、Takeda Akiko
    • Journal Title

      Journal of Machine Learning Research

      Volume: 22(115) Pages: 1-36

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] On Lp-hyperparameter Learning via Bilevel Nonsmooth Optimization2021

    • Author(s)
      Okuno Takayuki, Takeda Akiko, Kawana Akihiro, Watanabe Motokazu
    • Journal Title

      Journal of Machine Learning Research

      Volume: 22(245) Pages: 1-347

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Projected Gradient Method for Opinion Optimization with Limited Changes of Susceptibility to Persuasion2021

    • Author(s)
      Naoki Marumo, Atsushi Miyauchi, Akiko Takeda, Akira Tanaka
    • Journal Title

      CIKM

      Volume: 30 Pages: 1274-1283

    • DOI

      10.1145/3459637.3482408

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A Gradient Method for Multilevel Optimization2021

    • Author(s)
      Sato Ryo, Tanaka Mirai, Takeda Akiko
    • Journal Title

      Advances in Neural Information Processing Systems

      Volume: 34 Pages: 7522-7533

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Estimation of Gaussian Mixture Models via Tensor Moments with Application to Online Learning2020

    • Author(s)
      Donya Rahmani, Mahesan Niranjan, Damien Fay, Akiko Takeda, Jacek Brodzki
    • Journal Title

      Pattern Recognition Letters

      Volume: 131 Pages: 285-292

    • DOI

      10.1016/j.patrec.2020.01.001

    • Related Report
      2020 Annual Research Report 2019 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Theory and Algorithms for Shapelet-Based Multiple-Instance Learning2020

    • Author(s)
      Suehiro Daiki、Hatano Kohei、Takimoto Eiji、Yamamoto Shuji、Bannai Kenichi、Takeda Akiko
    • Journal Title

      Neural Computation

      Volume: 32 Issue: 8 Pages: 1580-1613

    • DOI

      10.1162/neco_a_01297

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Controllability maximization of large-scale systems using projected gradient method2020

    • Author(s)
      Kazuhiro Sato, Akiko Takeda
    • Journal Title

      IEEE Control Systems Letters

      Volume: 4 Pages: 821-826

    • DOI

      10.1109/lcsys.2020.2993983

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Data-driven structured noise filtering via common dynamics estimation2020

    • Author(s)
      Ivan Markovsky, Tianxiang Liu, Akiko Takeda
    • Journal Title

      IEEE Transactions on Signal Processing

      Volume: 68 Pages: 3064-3073

    • DOI

      10.1109/tsp.2020.2993676

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] A hybrid penalty method for a class of optimization problems with multiple rank constraints2020

    • Author(s)
      Tianxiang Liu, Ivan Markovsky, Ting Kei Pong, Akiko Takeda
    • Journal Title

      SIAM Journal on Matrix Analysis and Applications

      Volume: 41 Issue: 3 Pages: 1260-1283

    • DOI

      10.1137/19m1269919

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Generalized Subdifferentials of Spectral Functions over Euclidean Jordan Algebras2020

    • Author(s)
      Lourenco Bruno F.、Takeda Akiko
    • Journal Title

      SIAM Journal on Optimization

      Volume: 30 Issue: 4 Pages: 3387-3414

    • DOI

      10.1137/19m1245001

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Convex fairness constrained model using causal effect estimators2020

    • Author(s)
      Hikaru Ogura, Akiko Takeda
    • Journal Title

      WWW '20: Companion Proceedings of the Web Conference 2020

      Volume: - Pages: 723-732

    • DOI

      10.1145/3366424.3383556

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Construction Methods of the Nearest Positive System2020

    • Author(s)
      Kazuhiro Sato, Akiko Takeda
    • Journal Title

      IEEE Control Systems Letters

      Volume: 4 Issue: 1 Pages: 97-102

    • DOI

      10.1109/lcsys.2019.2921838

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Robust Bayesian Model Selection for Variable Clustering with the Gaussian Graphical Model2020

    • Author(s)
      Daniel Andrade, Akiko Takeda, Kenji Fukumizu
    • Journal Title

      Statistics and Computing

      Volume: 30 Issue: 2 Pages: 351-376

    • DOI

      10.1007/s11222-019-09879-9

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems2019

    • Author(s)
      Tianxiang Liu, Ting Kei Pong, Akiko Takeda
    • Journal Title

      Mathematical Programming

      Volume: 176 Issue: 1-2 Pages: 339-367

    • DOI

      10.1007/s10107-018-1327-8

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] A refined convergence analysis of pDCAe with applications to simultaneous sparse recovery and outlier detection2019

    • Author(s)
      Liu Tianxiang、Pong Ting Kei、Takeda Akiko
    • Journal Title

      Computational Optimization and Applications

      Volume: 73 Issue: 1 Pages: 69-100

    • DOI

      10.1007/s10589-019-00067-z

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Algorithm 996: BBCPOP: A Sparse Doubly Nonnegative Relaxation?of Polynomial Optimization Problems?with Binary, Box and Complementarity Constraints2019

    • Author(s)
      Naoki Ito, Sunyoung Kim, Masakazu Kojima, Akiko Takeda and Kim-Chuan Toh
    • Journal Title

      ACM Transaction on Mathematical Software

      Volume: 45 Issue: 3 Pages: 1-26

    • DOI

      10.1145/3309988

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization2019

    • Author(s)
      Michael Metel, Akiko Takeda
    • Journal Title

      Proceedings of the 36th International Conference on Machine Learning

      Volume: 97 Pages: 4537-4545

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Subspace methods for multi-channel sum-of-exponentials common dynamics estimation2019

    • Author(s)
      Ivan Markovsky, Tianxiang Liu, Akiko Takeda
    • Journal Title

      2019 IEEE 58th Conference on Decision and Control (CDC)}, Nice, France, 2019, pp. 2672-2675.

      Volume: - Pages: 2672-2675

    • Related Report
      2019 Annual Research Report
  • [Presentation] Random subspace optimization methods for large-scale optimiztion problems2024

    • Author(s)
      Akiko Takeda
    • Organizer
      NUS Seminar, National University of Singapore (Singapore, Singapore)
    • Related Report
      2022 Annual Research Report
  • [Presentation] Bi/trilevel Optimization Approach for Hyperparameter Selection2023

    • Author(s)
      Akiko Takeda
    • Organizer
      The SIAM Conference on Optimization (OP23), The Sheraton Grand Seattle (Seattle, Washington, U.S.)
    • Related Report
      2022 Annual Research Report
  • [Presentation] Applying Random projection techniques to nonconvex optimization problems2023

    • Author(s)
      Akiko Takeda
    • Organizer
      Mini Workshop on Optimization, University of Southampton (Southampton, United Kimgdom)
    • Related Report
      2022 Annual Research Report
  • [Presentation] Difference-of-Convex Approach for Nonconvex Nonsmooth Optimization Problems2021

    • Author(s)
      Akiko Takeda
    • Organizer
      5th ZIB-RIKEN-IMI-ISM MODAL Workshop on Optimization, Data Analysis and HPC in AI
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] 不確実性下での最適化手法: ロバスト最適化法の紹介2020

    • Author(s)
      武田朗子
    • Organizer
      第 41 回 IBISML 研究会
    • Related Report
      2020 Annual Research Report
    • Invited
  • [Presentation] Deterministic and Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization2020

    • Author(s)
      Akiko Takeda
    • Organizer
      Variational Analysis and Optimisation Webinar series
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Group Lasso for Household Energy Consumption Prediction and Toward Nonconvex Regularizer2019

    • Author(s)
      Akiko Takeda
    • Organizer
      The PolyU AMA - RIKEN AIP Joint Workshop on Optimization and Machine Learning,
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] DC Formulations and Algorithms for Sparse Optimization Problems2019

    • Author(s)
      Akiko Takeda
    • Organizer
      the Sixth International Conference on Continuous Optimization (ICCOPT 2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Efficient DC algorithm for nonconvex sparse optimization problems2019

    • Author(s)
      Akiko Takeda
    • Organizer
      International Conference on Optimization: Techniques and Applications (NACA-ICOTA2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Efficient DC Algorithm for Nonconvex Nonsmooth Optimization Problems2019

    • Author(s)
      Akiko Takeda
    • Organizer
      Faculty Seminar, Southwest Jiaotong University
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] 世の中の「困った」を解決する数学:数理最適化法の紹介2019

    • Author(s)
      武田朗子
    • Organizer
      RDC展 2019, 東芝研究開発センター
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Remarks] Publication List

    • URL

      https://www.or.mist.i.u-tokyo.ac.jp/takeda/publication-e.html

    • Related Report
      2022 Annual Research Report
  • [Remarks] Homepage of Akiko Takeda

    • URL

      https://www.or.mist.i.u-tokyo.ac.jp/takeda/publication-e.html

    • Related Report
      2020 Annual Research Report
  • [Remarks] Akiko Takeda's webpage

    • URL

      https://www.or.mist.i.u-tokyo.ac.jp/takeda/index-e.html

    • Related Report
      2019 Annual Research Report

URL: 

Published: 2019-04-18   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi