Difference between marginal likelihood and generalization error as statistical model evaluation based on algebraic geometry and structure learning theory
| Project/Area Number |
15K00331
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Research Field |
Soft computing
|
|---|
| Research Institution | Tokyo Institute of Technology |
|---|
Principal Investigator |
WATANABE SUMIO 東京工業大学, 情報理工学院, 教授 (80273118)
|
|---|
| Project Period (FY) |
2015-04-01 – 2020-03-31
|
| Project Status |
Completed (Fiscal Year 2019)
|
| Budget Amount *help |
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2019: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2018: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2017: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2016: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2015: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
|
|---|
| Keywords | 情報量規準 / 交差検証 / 周辺尤度 / WAIC / WBIC / 汎化誤差 / 自由エネルギー / 実対数閾値 / 漸近挙動 / ゼータ関数 / クロスバリデーション / 汎化損失 / 双有理不変量 / 交差損失 / 情報量規準WAIC / 非負値行列分解 / 汎関数分散 / 代数幾何 |
|---|
| Outline of Final Research Achievements |
In Bayesian statistics, two criteria, the generalization error and the marginal likelihood are well known for optimizing a model and a prior, however, their difference was not clarified.In this research, the following four results were obtained, (1) If the posterior can be approximated by a normal distribution, the optimal hyperparameters for WAIC and cross validation converge to the parameter that minimizes the average generalization error, whereas the parameter which maximizes the marginal likelihood does not. (2) Nonngeative matrix factorization is an example of singular statistical models, whose asymptotic generalization error and marginal likelihood were clarified. (3) its variational free energy was also clarified, (4) The asymptotic ditribution of the constant order term of the free energy, which is necessary to construct the most powerful test in Bayesian statistics, was clarified.
|
| Academic Significance and Societal Importance of the Research Achievements |
データの分析において統計モデルと事前分布をどのように設計したらよいかという問題は実社会においても常に必要になる課題である。この課題に対して情報量規準、交差検証、周辺尤度は、既に広く実用に用いられているが、それらの相違については必ずしも明確にはされていなかった。本研究により、次のことが明らかになった。(1)WAICと交差検証の最小化は平均汎化誤差を最小にするが、周辺尤度最大化ではそうならない。(2)非負値行列分解における汎化誤差と周辺尤度の漸近挙動の上界の値を求めることができる。(3)そのモデルの変分自由エネルギーの値を知ることができる。(4)混合正規分布のベイズ最強検定を作ることができる。
|
Report
(6 results)
Research Products
(26 results)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
[Presentation] ベイズ推論2016
Author(s)
渡邊澄夫
Organizer
電子情報通信学会ソサイエティ大会
Place of Presentation
北海道大学
Year and Date
2016-09-20
Related Report
Invited
-
-
-
-
-
-
-
-
