• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Statistical mechanics of heuristic methods in multi-stage learning

Research Project

Project/Area Number 21K21310
Research Category

Grant-in-Aid for Research Activity Start-up

Allocation TypeMulti-year Fund
Review Section 1002:Human informatics, applied informatics and related fields
Research InstitutionThe University of Tokyo

Principal Investigator

Takahashi Takashi  東京大学, 大学院理学系研究科(理学部), 助教 (90906661)

Project Period (FY) 2021-08-30 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥2,990,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥690,000)
Fiscal Year 2022: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2021: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Keywords統計力学 / 半教師あり学習 / レプリカ法 / 自己学習 / 疑似ラベル / アンサンブル学習 / 近似確率伝搬法 / 機械学習 / モデル圧縮
Outline of Research at the Start

本研究では、半教師あり学習やモデル圧縮などの、多段階の学習プロセスに基づくヒューリスティクス法の性質を統計物理学の手法を用いて解析する。特にFranz-Parisiポテンシャルの計算技法との関連に注目して解析を行う。これにより、どのような場面でどの程度これらのヒューリスティクス法が有用であるかを系統的/定量的に明らかにし、分析方針の策定や分析結果の解釈の場面でデータ分析者の参照に耐える理論的知見を構築することを目指す。

Outline of Final Research Achievements

The purpose of this study was to understand the behavior of a self-training algorithm that assigns pseudo labels to data points based on a pre-trained model and then retrains a new model using these labels. To achieve this, we analyzed the behavior of a linear model trained with this method for a binary classification problem using the mean-field theory of statistical mechanics. This analysis showed that the optimal approach varies with the number of iterations, and we organized strategies for efficiently improving generalization performance using this algorithm.

Academic Significance and Societal Importance of the Research Achievements

データ科学の実務的な場面ではラベル付きデータが十分にあるという理想的な設定にないことは一般的で、分析者は様々なヒューリスティクスを用いて問題に対処している。そのなかで多段階の学習に基づくヒューリスティクスはアイディアを実装に繋げやすく、近年盛んに用いられている。本研究はその流れに沿い、そのような多段階の学習に基づくヒューリスティクスの利用方法に関する示唆を与えるものであり、実務に向けた理論的知見を新たに加えたという意義があると思われる。

Report

(4 results)
  • 2023 Annual Research Report   Final Research Report ( PDF )
  • 2022 Research-status Report
  • 2021 Research-status Report
  • Research Products

    (12 results)

All 2024 2023 2022 2021

All Journal Article (3 results) (of which Int'l Joint Research: 3 results,  Peer Reviewed: 3 results) Presentation (9 results) (of which Int'l Joint Research: 3 results,  Invited: 1 results)

  • [Journal Article] Compressed Sensing Radar Detectors Under the Row-Orthogonal Design Model: A Statistical Mechanics Perspective2023

    • Author(s)
      Na Siqi、Huang Tianyao、Liu Yimin、Takahashi Takashi、Kabashima Yoshiyuki、Wang Xiqin
    • Journal Title

      IEEE Transactions on Signal Processing

      Volume: 71 Pages: 2668-2682

    • DOI

      10.1109/tsp.2023.3297743

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Role of Bootstrap Averaging in Generalized Approximate Message Passing2023

    • Author(s)
      Takahashi Takashi
    • Journal Title

      IEEE International Symposium on Information Theory (ISIT)

      Volume: 2023 Pages: 767-772

    • DOI

      10.1109/isit54713.2023.10206490

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Average case analysis of Lasso under ultra sparse conditions2023

    • Author(s)
      Koki Okajima, Xiangming Meng, Takashi Takahashi, Yoshiyuki Kabashima
    • Journal Title

      Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR

      Volume: 206 Pages: 11317-11330

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Presentation] 交互最適化法のダイナミクスについて2024

    • Author(s)
      岡島光希, 髙橋昂
    • Organizer
      日本物理学会2024年春季大会
    • Related Report
      2023 Annual Research Report
  • [Presentation] 疑似ラベルの構成法について2024

    • Author(s)
      髙橋昂
    • Organizer
      日本物理学会2024年春季大会
    • Related Report
      2023 Annual Research Report
  • [Presentation] Exploring bagging with structured data: Insights from precise asymptotics2023

    • Author(s)
      Takashi Takahashi
    • Organizer
      Workshop on Learning and Inference from Structured Data: Universality, Correlations and Beyond | (smr 3850)
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] A Statistical Mechanics Analysis of Iterative Self-Training2023

    • Author(s)
      Takashi Takahashi
    • Organizer
      STATPHYS28
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 高次元モデルにおける不均衡データ分類2023

    • Author(s)
      髙橋昂
    • Organizer
      日本物理学会第78回年次大会
    • Related Report
      2023 Annual Research Report
  • [Presentation] Sharp Asymptotics of Self-training with Linear Classifier2022

    • Author(s)
      Takashi Takahashi
    • Organizer
      Youth in High-Dimensions: Recent Progress in Machine Learning, High-Dimensional Statistics and Inference
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] 反復型自己学習アルゴリズムのレプリカ解析2022

    • Author(s)
      髙橋昂
    • Organizer
      日本物理学会2022年秋季大会 2022年9月14日
    • Related Report
      2022 Research-status Report
  • [Presentation] ブートストラップ平均化された不偏推定量の統計力学的解析2022

    • Author(s)
      髙橋昂
    • Organizer
      日本物理学会2023年春季大会 2023年3月25日
    • Related Report
      2022 Research-status Report
  • [Presentation] 半教師あり学習の平衡統計力学的解析2021

    • Author(s)
      髙橋昂
    • Organizer
      日本物理学会2021年秋季大会
    • Related Report
      2021 Research-status Report

URL: 

Published: 2021-10-22   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi