• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Statistical Mechanical Analysis of non convex sparse penalty

Research Project

Project/Area Number 16K16131
Research Category

Grant-in-Aid for Young Scientists (B)

Allocation TypeMulti-year Fund
Research Field Soft computing
Research InstitutionThe Institute of Statistical Mathematics

Principal Investigator

Sakata Ayaka  統計数理研究所, 数理・推論研究系, 助教 (80733071)

Research Collaborator Xu Yingying  
Obuchi Tomoyuki  
Project Period (FY) 2016-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥3,510,000 (Direct Cost: ¥2,700,000、Indirect Cost: ¥810,000)
Fiscal Year 2018: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2017: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2016: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Keywordsスパース推定 / 非凸正則化 / 統計物理 / 圧縮センシング / 交差検証法 / 非凸スパース正則化 / 確率伝搬法 / レプリカ法 / 相転移 / 情報量規準 / スパース正則化 / 情報量基準
Outline of Final Research Achievements

It has been implied that sparse estimation based on nonconvex penalties leads high compression performance compared with that based on convex penalties. However, the nonconvex penalties provide lots of local minima depending on the values of regularization parameters. Mathematical methods that derive the condition for the appearance of local minima and algorithms to solve problems penalized by nonconvex penalty are required. In this study, we identified the parameter regions where the local minima appear based on the statistical mechanical method. Further, we show that the estimation problems with nonconvex penalties can be solved with the same computational cost as convex penalties. In addition, we proposed model selection methods.

Academic Significance and Societal Importance of the Research Achievements

非凸正則化は高いポテンシャルを持つものの、数学的に扱いにくいという問題点があった。本研究では、その数学的困難を統計力学的手法により解決し、凸正則化と同様に扱うことができる条件を特定した。また、収束条件を導出することができる確率伝搬法を非凸正則化に導入したことで、適切な正則化パラメータの選び方が明らかになった。また低コストで予測誤差を評価することが可能となり、予測に基づくモデリング方法も整った。これらの研究結果は、非凸正則化を用いた新しい推定方法を広く利用可能とするものであり、データ駆動科学に大きく貢献すると考える。

Report

(4 results)
  • 2018 Annual Research Report   Final Research Report ( PDF )
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (18 results)

All 2018 2017 2016 Other

All Int'l Joint Research (2 results) Journal Article (4 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 4 results,  Open Access: 4 results,  Acknowledgement Compliant: 2 results) Presentation (12 results) (of which Int'l Joint Research: 2 results)

  • [Int'l Joint Research] Aalto University(フィンランド)

    • Related Report
      2018 Annual Research Report
  • [Int'l Joint Research] Aalto university(フィンランド)

    • Related Report
      2017 Research-status Report
  • [Journal Article] Estimator of prediction error based on approximate message passing for penalized linear regression2018

    • Author(s)
      Ayaka Sakata
    • Journal Title

      Journal of Statistical Mechanics: Theory and Experiment

      Volume: 2018 Issue: 6 Pages: 063404-063404

    • DOI

      10.1088/1742-5468/aac910

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis2017

    • Author(s)
      Ayaka Sakata and Yingying Xu
    • Journal Title

      Journal of Statistical Mechanics: Theory and Experiment

      Volume: 2017 Issue: 3 Pages: 033404-033404

    • DOI

      10.1088/1742-5468/aab051

    • Related Report
      2018 Annual Research Report 2017 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Phase transitions and sample complexity in Bayes-optimal matrix factorization2016

    • Author(s)
      Y. Kabashima, F. Krzakala, M. Mezard, A. Sakata, and L. Zdeborova
    • Journal Title

      IEEE Transactions on Information Theory

      Volume: 62(7) Issue: 7 Pages: 4228-4265

    • DOI

      10.1109/tit.2016.2556702

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research / Acknowledgement Compliant
  • [Journal Article] Evaluation of Generalized Degrees of Freedom for Sparse Estimation by Replica Method2016

    • Author(s)
      Ayaka Sakata
    • Journal Title

      Journal of Statistical Mechanics: Theory and Experiment

      Volume: 2016 Issue: 12 Pages: 1-29

    • DOI

      10.1088/1742-5468/2016/12/123302

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access / Acknowledgement Compliant
  • [Presentation] 非凸正則化付き線形回帰における復号性能2018

    • Author(s)
      坂田綾香
    • Organizer
      日本物理学会2018年秋季大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Model selection under SCAD and MCP based on approximate message passing2018

    • Author(s)
      坂田綾香
    • Organizer
      2018年度統計関連学会連合大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] スパース推定におけるモデル選択規準2018

    • Author(s)
      坂田綾香
    • Organizer
      SITA2018
    • Related Report
      2018 Annual Research Report
  • [Presentation] 非凸性制御下での非凸制約最小化による信号復元2018

    • Author(s)
      坂田綾香
    • Organizer
      最適化:モデリングとアルゴリズム
    • Related Report
      2018 Annual Research Report
  • [Presentation] Cross validation in sparse linear regression with piecewise nonconvex penalties and its acceleration2018

    • Author(s)
      Ayaka Sakata
    • Organizer
      The 4th ISM-ZIB-IMI workshop on mathematical optimization and data analysis
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 非凸スパース正則化に対する確率伝搬法と収束条件2018

    • Author(s)
      坂田綾香
    • Organizer
      日本物理学会第73回年次大会
    • Related Report
      2017 Research-status Report
  • [Presentation] LASSOにおけるAICの揺らぎ2017

    • Author(s)
      坂田綾香
    • Organizer
      日本物理学会
    • Place of Presentation
      大阪大学
    • Year and Date
      2017-03-17
    • Related Report
      2016 Research-status Report
  • [Presentation] Fluctuation of AIC in LASSO and its approximation by bootstrap method2017

    • Author(s)
      坂田綾香
    • Organizer
      2017年統計関連学会連合大会
    • Related Report
      2017 Research-status Report
  • [Presentation] L1正則化におけるAICのブートストラップ補正と分布の評価2017

    • Author(s)
      坂田綾香
    • Organizer
      日本物理学会2017年秋季大会
    • Related Report
      2017 Research-status Report
  • [Presentation] 非凸スパース正則化に対する確率伝搬法とその収束条件2017

    • Author(s)
      坂田綾香
    • Organizer
      IBIS2017
    • Related Report
      2017 Research-status Report
  • [Presentation] Approximate message passing for nonconvex sparse regularization and its convergence condition2017

    • Author(s)
      Ayaka Sakata
    • Organizer
      ISI-ISM-ISSAS joint conference
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] スパース正則化におけるモデル選択規準の評価2016

    • Author(s)
      坂田綾香
    • Organizer
      日本物理学会
    • Place of Presentation
      金沢大学
    • Year and Date
      2016-09-13
    • Related Report
      2016 Research-status Report

URL: 

Published: 2016-04-21   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi