• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Establishment of Algebraic Geometrical Methods in Statistical Learning Theory

Research Project

Project/Area Number 15500130
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeSingle-year Grants
Section一般
Research Field Sensitivity informatics/Soft computing
Research InstitutionTokyo Institute of Technology

Principal Investigator

WATANABE Sumio  Tokyo Institute of Technology, P&I Lab, Professor, 精密工学研究所, 教授 (80273118)

Co-Investigator(Kenkyū-buntansha) 山崎 啓介  東京工業大学, 精密工学研究所, 助手 (60376936)
Project Period (FY) 2003 – 2006
Project Status Completed (Fiscal Year 2006)
Budget Amount *help
¥2,900,000 (Direct Cost: ¥2,900,000)
Fiscal Year 2006: ¥600,000 (Direct Cost: ¥600,000)
Fiscal Year 2005: ¥900,000 (Direct Cost: ¥900,000)
Fiscal Year 2004: ¥600,000 (Direct Cost: ¥600,000)
Fiscal Year 2003: ¥800,000 (Direct Cost: ¥800,000)
KeywordsAlgebraic Geometry / Statistical Learning Theory / Zeta function / Learning System / Learning Machine / Learning Theory / Algebraic Analysis / Information System / 特異モデル / 確率的複雑さ / ベイズ法 / 変分ベイズ法 / ゼータ函数 / 縮小ランク回帰 / 文脈自由文法 / 汎化誤差 / カルバック情報量 / ブローアップ / 特定不能
Research Abstract

A lot of learning machines such as artificial neural networks, normal mixtures, hidden Markov models, Boltzmann machines, Bayesian networks, reduced rank regressions are nonidentifiable and singular learning machines. They have degenerate Fisher Information matrices resulting that the conventional statistical learning theory does not hold. In order to establish the new learning theory, we need algebraic geometrical method to treat the likelihood functions. In this research, we have the following results. (1) The fundamental form of the likelihood function were introduced based on desingularization problem. (2) The asymptotic form of the marginal likelihood were derived based on the zeta function of learning theory. (3) The asymptotic behavior of the variational Bayes or the mean field approximation were obtained. (4) The asymptotic behavior of the empirical Bayes appriximation were obtained. These are the mathematical foundation originally established in this research.

Report

(5 results)
  • 2006 Annual Research Report   Final Research Report Summary
  • 2005 Annual Research Report
  • 2004 Annual Research Report
  • 2003 Annual Research Report
  • Research Products

    (38 results)

All 2006 2005 2004 Other

All Journal Article (29 results) Book (3 results) Publications (6 results)

  • [Journal Article] Stochastic complexities of gaussian mixtures in variational bayesian approximation2006

    • Author(s)
      K Watanabe, S Watanabe
    • Journal Title

      Journal of Machine Learning Research Vol.7

      Pages: 625-644

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Generalization Performance of Subspace Bayes Approach in Linear Neural Networks2006

    • Author(s)
      S Nakajima, S Watanabe
    • Journal Title

      IEICE Transactions Vol.E89-D, no.3

      Pages: 1128-1138

    • NAID

      110004719390

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 完全2部グラフ型ボルツマンマシンの平均場近似による確率的複雑さについて2006

    • Author(s)
      西山 悠, 渡辺澄夫
    • Journal Title

      電子情報通信学会論文誌 Vol.J89-A, no.8

      Pages: 671-678

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 局所化ベイズ学習法2006

    • Author(s)
      高松慎吾, 中島伸一, 渡辺澄夫
    • Journal Title

      電子情報通信学会論文誌 Vol.J89-D-II, no.10

      Pages: 2260-2268

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 隠れマルコフモデルの変分ベイズ学習における確率的複雑さについて2006

    • Author(s)
      星野力, 渡辺一帆, 渡辺澄夫
    • Journal Title

      電子情報通信学会論文誌 Vol.J89-D-II, No.6

      Pages: 1279-1287

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Variational bayesian stochastic complexity of mixture models2006

    • Author(s)
      K Watanabe, S Watanabe
    • Journal Title

      Proc. of Advances in Neural Information Processing Systems Vol.18

      Pages: 1465-1472

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Generalization Performance of Subspace Bayes Approach in Linear Neural Networks2006

    • Author(s)
      S Nakajima, S Watanabe
    • Journal Title

      IEICE Transactions vol.89-E-D

      Pages: 1128-1138

    • NAID

      110004719390

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Stocahstic complexity of complete bipartite Boltzmann machine in mean field approximation2006

    • Author(s)
      Yu Nichiyama., S.Watanabe
    • Journal Title

      IEICE Transactions Vol.89-A

      Pages: 671-678

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] locally Bayes learning,2006

    • Author(s)
      Shingo Tanakamatsu, Shinichi Nakajima, Sumio Watanabe
    • Journal Title

      IEICE Transactions Vol.J89-D-II

      Pages: 2260-2268

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Stochastic complexity of hidden markov model in Variational Bayes learning.2006

    • Author(s)
      Tikara Hosino, Kazuho Watanabe, Sumio Watanabe
    • Journal Title

      IEICE Transactions Vol.J89-D-II

      Pages: 1279-1287

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Variational bayesian stochastic complexity of mixture models2006

    • Author(s)
      K Watanabe, S Watanabe
    • Journal Title

      Proc. of Advances in Neural Information Processing Systems vol.18

      Pages: 1465-1472

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Stochastic complexities of gaussian mixtures in variational bayesian approximation2006

    • Author(s)
      K Watanabe, S Watanabe
    • Journal Title

      Journal of Machine Learning Research Vol. 7

      Pages: 625-644

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Generalization Performance of Subspace Bayes Approach in Linear Neural Networks2006

    • Author(s)
      S Nakajima, S Watanabe
    • Journal Title

      IEICE Transactions Vol. E89-D, no. 3

      Pages: 1128-1138

    • NAID

      110004719390

    • Related Report
      2006 Annual Research Report
  • [Journal Article] 完全2部グラフ型ボルツマンマシンの平均場近似による確率的複雑さについて2006

    • Author(s)
      西山 悠, 渡辺澄夫
    • Journal Title

      電子情報通信学会論文誌 Vol. J89-A, no. 8

      Pages: 671-678

    • Related Report
      2006 Annual Research Report
  • [Journal Article] 局所化ベイズ学習法2006

    • Author(s)
      高松慎吾, 中島伸一, 渡辺澄夫
    • Journal Title

      電子情報通信学会論文誌 vol. J89-D-II, no. 10

      Pages: 2260-2268

    • Related Report
      2006 Annual Research Report
  • [Journal Article] 隠れマルコフモデルの変分ベイズ学習における確率的複雑さについて2006

    • Author(s)
      星野力, 渡辺一帆, 渡辺澄夫
    • Journal Title

      電子情報通信学会論文誌 Vol. J89-D-II,No. 6

      Pages: 1279-1287

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Variational bayesian stochastic complexity of mixture models2006

    • Author(s)
      K Watanabe, S Watanabe
    • Journal Title

      Proc. of Advances in Neural Information Processing Systems Vol. 18

      Pages: 1465-1472

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Stochastic complexities of reduced rank regression in Bayesian estimation2005

    • Author(s)
      M.Aoyagi, S.Watanabe
    • Journal Title

      International Journal of Neural Networks 18・7

      Pages: 924-933

    • Related Report
      2005 Annual Research Report
  • [Journal Article] カルバック情報量の分割による特異モデルの汎化誤差計算法2005

    • Author(s)
      永田賢二, 渡辺澄夫
    • Journal Title

      電子情報通信学会誌 J88-D-II・6

      Pages: 994-1002

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Algebraic geometry and stochastic complexity of hidden Markov models2005

    • Author(s)
      K.Yamazaki, S.Watanabe
    • Journal Title

      Neurocomputing 69

      Pages: 62-84

    • Related Report
      2005 Annual Research Report
  • [Journal Article] 特異点解消とニューラルネットワークのベイズ推定における汎化誤差2005

    • Author(s)
      青柳美輝, 渡辺澄夫
    • Journal Title

      電子情報通信学会誌 J88-D-II・10

      Pages: 2112-2124

    • Related Report
      2005 Annual Research Report
  • [Journal Article] 特異モデルにおけるマルコフ連鎖モンテカルロ法の評価法2005

    • Author(s)
      中野修弘, 高橋克之, 渡辺澄夫
    • Journal Title

      電子情報通信学会誌 J88-D-II

      Pages: 2011-2020

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Algebraic geometry of singular learning machines and symmetry of generalization and training errors2005

    • Author(s)
      Sumio Watanabe
    • Journal Title

      Neurocomputing 67

      Pages: 198-213

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Algebraic geometry of singular learning machines and symmetry of generalization errors and training errors2005

    • Author(s)
      渡辺澄夫
    • Journal Title

      Neurocomputing (印刷中)

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 特異モデルの学習理論2005

    • Author(s)
      渡辺澄夫, 福水健次, 萩原克幸, 甘利俊一
    • Journal Title

      電子情報通信学会誌 Vol.J88-D-II

      Pages: 159-169

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Singularities in complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities2005

    • Author(s)
      山崎啓介, 渡辺澄夫
    • Journal Title

      IEEE Transactions on Neural Networks (印刷中)

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Algebraic geometry and stochastic complexity of hidden Markov models2005

    • Author(s)
      山崎啓介, 渡辺澄夫
    • Journal Title

      Neurocomputing (印刷中)

    • Related Report
      2004 Annual Research Report
  • [Journal Article] カルバック情報量の分割による特異モデルの汎化誤差計算法2005

    • Author(s)
      永田賢二, 渡辺澄夫
    • Journal Title

      電子情報通信学会D-II (印刷中)

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 人工知能と特異な学習モデル2004

    • Author(s)
      渡辺澄夫
    • Journal Title

      人工知能学会誌 Vol.19,No.6

      Pages: 637-641

    • Related Report
      2004 Annual Research Report
  • [Book] 代数幾何と学習理論2006

    • Author(s)
      渡辺澄夫
    • Total Pages
      200
    • Publisher
      森北出版
    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Annual Research Report 2006 Final Research Report Summary
  • [Book] 学習システムの理論と実現2005

    • Author(s)
      渡邊澄夫, 萩原克幸, 赤穂昭太郎, 本村陽一, 福水健次, 岡田真人, 青柳美輝
    • Total Pages
      195
    • Publisher
      森北出版
    • Related Report
      2005 Annual Research Report
  • [Book] 確率と統計-情報学への架橋-2005

    • Author(s)
      渡辺澄夫, 村田昇
    • Total Pages
      180
    • Publisher
      コロナ社
    • Related Report
      2004 Annual Research Report
  • [Publications] S.Watanabe, S.Amari: "Learning Coefficients of Layered Models when the true parameter mismatches the singularities"Neural Computation. 15・5. 1013-1033 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] K.Yamazaki, S.Watanabe: "Singularities in mixture models and upper bounds of stochastic complexity"International Journal of Neural Networks. 16・7. 1029-1038 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Keisuke Yamazaki, Sumio Watanabe: "Stochastic complexity of Bayesian networks"Uncertainty in Artificial Intelligence. 592-599 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Keisuke Yamazaki, Sumio Watanabe: "Stochastic complexities of hidden Markov models"Neural Networks for Signal Processing. (CD-ROM). (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] Sumio Watanabe, Keisuke Yamazaki: "Algebraic geometry, zeta functions, and marginal likelihood of Bayesian statistics"Anniversary of the information criterion. (CD-ROM). (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] 渡辺澄夫: "特異モデルとベイズ学習"日本神経回路学会誌. 10・4. 211-219 (2003)

    • Related Report
      2003 Annual Research Report

URL: 

Published: 2003-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi