Information Theoretic Interpretation and Design of Generalized Bayesian Learning Methods

2023 Fiscal Year Final Research Report

• Project/Area Number: 19K11825
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 60010:Theory of informatics-related
• Research Institution: Toyohashi University of Technology
• Principal Investigator: Watanabe Kazuho 豊橋技術科学大学, 工学(系)研究科(研究院), 准教授 (10506744)
• Project Period (FY): 2019-04-01 – 2024-03-31
• Keywords: レート歪み関数 / 板倉・斎藤距離 / オンライン予測 / 局所変分近似 / 不感応パラメータ / 正則化パラメータ / L1トレンドフィルタ

Outline of Final Research Achievements

To establish a fundamental theory for the information theoretically principled design of statistical learning methods, we studied online time-series prediction, the optimal reconstruction distribution achieving the rate-distortion function, the estimation of the insensitivity in loss functions and the estimation of the regularization parameter in sparse estimation methods. For respective subproblems, we extended a minimax optimal prediction method for real valued data to the prediction of the time-varying probabilities from binary inputs, characterized the optimal reconstruction distribution achieving the rate-distortion function for Itakura-Saito distortion measure, analyzed the generalization errors in learning of the insensitive parameter, and developed an efficient approximate estimation method for the regularization parameter of L1 trend filtering, which extracts piece-wise linear trend from time-series data.

Free Research Field

統計的学習理論

Academic Significance and Societal Importance of the Research Achievements

歪み有りデータ圧縮の限界であるレート歪み関数を達成する再構成分布を特徴づける例を追加しており、既存の結果との対比に用いることができる。再構成分布の最適化に対応する、ベイズの定理に基づく学習法は一般に計算困難性を伴う。具体的な時系列解析問題において効率的な近似法を構築し、その性質が実験的に、または一部理論的に明らかにされた。