• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

The Minimum Description Length Principle and Learning Theory

Research Project

Project/Area Number 24500018
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Fundamental theory of informatics
Research InstitutionKyushu University

Principal Investigator

TAKEUCHI Junichi  九州大学, システム情報科学研究科(研究院, 教授 (80432871)

Co-Investigator(Renkei-kenkyūsha) ONO Hirotaka  九州大学, 経済学研究院, 准教授 (00346826)
Research Collaborator BARRON Andrew R.  Yale University, Dept. of Statistics, 教授
Project Period (FY) 2012-04-01 – 2015-03-31
Project Status Completed (Fiscal Year 2014)
Budget Amount *help
¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000)
Fiscal Year 2014: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2013: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000)
Fiscal Year 2012: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000)
Keywords記述長最小原理 / MDL / 確率的コンプレキシティ / 指数型分布族 / 木情報源モデル / 局所指数族バンドル / Jeffreys事前分布 / 木情報源 / Markovモデル / スパース重ね合わせ符号 / MDL原理 / Bayes符号 / ガウス通信路 / 混合型分布族 / ポートフォリオ
Outline of Final Research Achievements

Concerning the minimum description length (MDL) principle, we performed the studies on evaluation of the stochastic complexity (SC) and the data compression and sequential prediction strategies which achieves the SC for various target models. Those algorithms performs as the minimax strategies with respect to logarithmic or coding regret. We obtained the minimax strategies when the target model is the i.i.d. general smooth families of probability distributions. Our strategy is a combination of the mixture of the target class with Jeffreys prior and a mixture of the local exponential family bundle of the target class. In particular, we succeeded to remove the restriction on the set of data sequences for the mixture family case and tree model case. We also studied sparse superposition code (Barron et al. 2011-), which is one of capacity achieving codes.

Report

(4 results)
  • 2014 Annual Research Report   Final Research Report ( PDF )
  • 2013 Research-status Report
  • 2012 Research-status Report
  • Research Products

    (28 results)

All 2015 2014 2013 2012 Other

All Journal Article (3 results) (of which Peer Reviewed: 3 results,  Acknowledgement Compliant: 1 results) Presentation (22 results) (of which Invited: 4 results) Book (1 results) Remarks (2 results)

  • [Journal Article] Least Squares Superposition Codes with Bernoulli Dictionary are Still Reliable at Rates up to Capacity2014

    • Author(s)
      Y. Takeishi, M. Kawakita, & J. Takeuchi
    • Journal Title

      IEEE Transactions on Information Theory

      Volume: 60 Issue: 5 Pages: 2737-2750

    • DOI

      10.1109/tit.2014.2312728

    • Related Report
      2014 Annual Research Report 2013 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Safe Semi-supervised Learning Based on Weighted Likelihood2014

    • Author(s)
      M. Kawakita & J. Takeuchi
    • Journal Title

      Neural Networks

      Volume: 53 Pages: 146-164

    • DOI

      10.1016/j.neunet.2014.01.016

    • Related Report
      2013 Research-status Report
    • Peer Reviewed
  • [Journal Article] Properties of Jeffreys Mixture for Markov Sources2013

    • Author(s)
      J. Takeuchi, T. Kawabata & A. R. Barron
    • Journal Title

      IEEE transactions on Information Theory

      Volume: 59 Issue: 1 Pages: 438-457

    • DOI

      10.1109/tit.2012.2219171

    • Related Report
      2012 Research-status Report
    • Peer Reviewed
  • [Presentation] L1罰則付き線形回帰のMDLによる推定誤差上界について2015

    • Author(s)
      豊暉原侑心,川喜田雅則,竹内純一
    • Organizer
      電子情報通信学会情報理論研究会 (IT)
    • Place of Presentation
      福岡県北九州市
    • Year and Date
      2015-03-02 – 2015-03-03
    • Related Report
      2014 Annual Research Report
  • [Presentation] スパース構造学習を用いた異常検知によるボットネット検出実験2015

    • Author(s)
      向井 脩,川村勇気,川喜田雅則,竹内純一
    • Organizer
      電子情報通信学会情報セキュリティ研究会 (ISEC)
    • Place of Presentation
      福岡県北九州市
    • Year and Date
      2015-03-02
    • Related Report
      2014 Annual Research Report
  • [Presentation] スパース構造学習を用いたボットネット検出法の性能評価2015

    • Author(s)
      向井 脩,川村勇気,川喜田雅則,竹内純一
    • Organizer
      2015年暗号と情報セキュリティシンポジウム
    • Place of Presentation
      福岡県北九州市
    • Year and Date
      2015-01-20 – 2015-01-23
    • Related Report
      2014 Annual Research Report
  • [Presentation] 非負値行列因子分解におけるMDL原理について2014

    • Author(s)
      川村勇気,川喜田雅則,村田昇,竹内純一
    • Organizer
      第37回情報理論とその応用シンポジウム
    • Place of Presentation
      富山県黒部市
    • Year and Date
      2014-12-10 – 2014-12-12
    • Related Report
      2014 Annual Research Report
  • [Presentation] MDL原理入門2014

    • Author(s)
      竹内純一
    • Organizer
      第37回情報理論とその応用シンポジウム
    • Place of Presentation
      富山県黒部市
    • Year and Date
      2014-12-10 – 2014-12-12
    • Related Report
      2014 Annual Research Report
    • Invited
  • [Presentation] 圧縮センシングによる超解像におけるAMPアルゴリズムの振る舞いについて2014

    • Author(s)
      高木美里,三村和史,川喜田雅則,竹内純一
    • Organizer
      第37回情報理論とその応用シンポジウム
    • Place of Presentation
      富山県黒部市
    • Year and Date
      2014-12-10 – 2014-12-12
    • Related Report
      2014 Annual Research Report
  • [Presentation] Stochastic complexity for tree models2014

    • Author(s)
      J. Takeuchi & A. R. Barron
    • Organizer
      2014 IEEE Information Theory Workshop
    • Place of Presentation
      Hobart, Tasmania, Australia
    • Year and Date
      2014-11-02 – 2014-11-05
    • Related Report
      2014 Annual Research Report
  • [Presentation] A Generalized Sufficient Condition for Global Convergence of Modified Multiplicative Updates for NMF2014

    • Author(s)
      N. Takahashi, J. Katayama, & J. Takeuchi
    • Organizer
      2014 International Symposium on Nonlinear Theory and its Applications
    • Place of Presentation
      Luzern, Switzerland
    • Year and Date
      2014-09-14 – 2014-09-18
    • Related Report
      2014 Annual Research Report
  • [Presentation] Geometry of Markov Chains, Finite State Machines, and Tree Models2014

    • Author(s)
      J. Takeuchi
    • Organizer
      電子情報通信学会情報理論研究会 (IT)
    • Place of Presentation
      兵庫県神戸市
    • Year and Date
      2014-07-17 – 2014-07-18
    • Related Report
      2014 Annual Research Report
  • [Presentation] When is a Tree Model an Exponential Family?2014

    • Author(s)
      J. Takeuchi
    • Organizer
      The Seventh Workshop on Information Theoretic Methods in Science and Engineering(招待講演)
    • Place of Presentation
      Honolulu, HI, USA
    • Year and Date
      2014-07-05 – 2014-07-07
    • Related Report
      2014 Annual Research Report
    • Invited
  • [Presentation] Asymptotically minimax regret for models with hidden variables2014

    • Author(s)
      J. Takeuchi & A. R. Barron
    • Organizer
      2014 IEEE International Symposium on Information Theory
    • Place of Presentation
      Honolulu, HI, USA
    • Year and Date
      2014-06-29 – 2014-07-04
    • Related Report
      2014 Annual Research Report
  • [Presentation] スパース重ね合わせ符号の復号実験による性能評価2014

    • Author(s)
      宮本耕平,高木美里,川喜田雅則,竹内純一
    • Organizer
      電子情報通信学会情報理論研究会 (IT)
    • Place of Presentation
      大分県別府市
    • Year and Date
      2014-05-16
    • Related Report
      2014 Annual Research Report
  • [Presentation] Boundedness of modified multiplicative updates for nonnegative matrix factorization2013

    • Author(s)
      J. Katayama, N. Takahashi, & J. Takeuchi
    • Organizer
      The Fifth IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing
    • Place of Presentation
      Saint Martin, France
    • Related Report
      2013 Research-status Report
  • [Presentation] Asymptotically Minimax Regret by Bayes Mixtures for Non-exponential Families2013

    • Author(s)
      J. Takeuchi & A. R. Barron
    • Organizer
      2013 IEEE Information Theory Workshop
    • Place of Presentation
      Sevilie, Spain
    • Related Report
      2013 Research-status Report
  • [Presentation] Least Squares Superposition Codes with Bernoulli Dictionary are Still Reliable at Rates up to Capacity2013

    • Author(s)
      Y. Takeishi, M. Kawakita, & J. Takeuchi
    • Organizer
      2013 IEEE International Symposium on Information Theory
    • Place of Presentation
      Istanbul, Turkey
    • Related Report
      2013 Research-status Report
  • [Presentation] Asymptotically Minimax Prediction for Markov Sources2013

    • Author(s)
      J. Takeuchi
    • Organizer
      The Sixth Workshop on Information Theoretic Methods in Science and Engineering
    • Place of Presentation
      Tokyo, Japan
    • Related Report
      2013 Research-status Report
    • Invited
  • [Presentation] 圧縮センシングによるシングルフレーム超解像について2013

    • Author(s)
      高木美里,三村和史,竹内純一
    • Organizer
      第36回情報理論とその応用シンポジウム
    • Place of Presentation
      静岡県伊東市
    • Related Report
      2013 Research-status Report
  • [Presentation] 非負値行列分解を用いたボットネット検出実験2013

    • Author(s)
      川村勇気,島村隼平,中里純二,吉岡克成,衛藤将史,井上大介,竹内純一,中尾康二
    • Organizer
      電子情報通信学会情報通信システムセキュリティ研究会 (ICSS)
    • Place of Presentation
      福岡県福岡市
    • Related Report
      2013 Research-status Report
  • [Presentation] Botnet Detection based on Non-negative Matrix Factorization and the MDL Principle2012

    • Author(s)
      S. Yamauchi, M. Kawakita, & J. Takeuchi
    • Organizer
      the 19th International Conference on Neural Information Processing
    • Place of Presentation
      Doha, Qatar
    • Related Report
      2012 Research-status Report
  • [Presentation] High Dimensional Data Analysis for Botnet Detection2012

    • Author(s)
      J. Takeuchi
    • Organizer
      the 19th International Conference on Neural Information Processing
    • Place of Presentation
      Doha, Qatar
    • Related Report
      2012 Research-status Report
    • Invited
  • [Presentation] Information Theoretic Limit of Single Frame Superresolution2012

    • Author(s)
      K. Yamaguchi, M. Kawakita, N. Takahashi, & J. Takeuchi
    • Organizer
      The Third International Conference on Emerging Security Technologies
    • Place of Presentation
      Lisbon, Portugal
    • Related Report
      2012 Research-status Report
  • [Presentation] Constant Markov Portfolio and Its Application to Universal Portfolio with Side Information2012

    • Author(s)
      M. Tsurusaki & J. Takeuchi
    • Organizer
      2012 IEEE International Symposium on Information Theory
    • Place of Presentation
      Boston, USA
    • Related Report
      2012 Research-status Report
  • [Book] A Mathematical Approach to Research Problems of Science and Technology (章題 An introduction of the minimum description length principle)2014

    • Author(s)
      J. Takeuchi: (book chapter (分担執筆))
    • Total Pages
      18
    • Publisher
      Springer
    • Related Report
      2014 Annual Research Report
  • [Remarks] Jun'ichi Takeuchi's Page

    • URL

      http://www-kairo.csce.kyushu-u.ac.jp/~tak/

    • Related Report
      2012 Research-status Report
  • [Remarks] 情報回路及び信号処理研究室

    • URL

      http://www-kairo.csce.kyushu-u.ac.jp/homepage.ja.html

    • Related Report
      2012 Research-status Report

URL: 

Published: 2013-05-31   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi