Information-Theoretic View and Development of Kernel-Based Learning Methods

• Project/Area Number: 23700175
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Multi-year Fund
• Research Field: Intelligent informatics
• Research Institution: Toyohashi University of Technology (2014); Nara Institute of Science and Technology (2011-2013)
• Principal Investigator: WATANABE Kazuho 豊橋技術科学大学, 工学(系)研究科(研究院), 講師 (10506744)
• Project Period (FY): 2011-04-28 – 2015-03-31
• Project Status: Completed (Fiscal Year 2014)
• Budget Amount: ¥3,900,000 (Direct Cost: ¥3,000,000、Indirect Cost: ¥900,000); Fiscal Year 2014: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000); Fiscal Year 2013: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000); Fiscal Year 2012: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2011: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
• Keywords: 混合モデル / レート歪み関数 / 再構成分布 / イプシロン不感応損失 / 漸近的ミニマックス性 / 変分ベイズ法 / 国際情報交換 / フィンランド / ガンマ分布 / 国際情報交換（フィンランド） / ノンパラメトリック推定 / 離散分布 / クラスタリング

Outline of Final Research Achievements

We developed a learning method of mixture models, which unifies existing methods such as convex clustering and kernel vector quantization. We provided an interpretation of this method as an optimization problem for the evaluation of the rate-distortion function. The rate-distortion function indicates the performance of optimal lossy data compression systems. We evaluated rate-distortion functions of practical loss functions such as absolute-log distortion and epsilon-insensitive distortion measures. We also evaluated the performance of an online prediction algorithm to provide an efficient method to approximate the optimal prediction algorithm.

Research Products

• [Journal Article] Entropic risk minimization for nonparametric estimation of mixing distributions (2015)
  • Author(s): Kazuho Watanabe and Shiro Ikeda
  • Journal Title: Machine Learning Volume: 99(1) Issue: 1 Pages: 119-136
  • DOI: 10.1007/s10994-014-5467-7
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Journal Article] Variational inference with ARD prior for NIRS diffuse optical tomography (2015)
  • Author(s): Atsushi Miyamoto, Kazuho Watanabe, Kazushi Ikeda, and Masa-aki Sato
  • Journal Title: IEEE Transactions on Neural Networks and Learning Systems Volume: 26(5) Issue: 5 Pages: 1109-1114
  • DOI: 10.1109/tnnls.2014.2328576
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Variational Bayesian inference algorithms for infinite relational model of network data (2015)
  • Author(s): Takuya Konishi, Takatomi Kubo, Kazuho Watanabe, and Kazushi Ikeda
  • Journal Title: IEEE Transactions on Neural Networks and Learning Systems Volume: to be assigned Issue: 9 Pages: 2176-2181
  • DOI: 10.1109/tnnls.2014.2362012
  • Peer Reviewed
• [Journal Article] Achievability of asymptotic minimax regret by horizon-dependent and horizon-independent strategies (2015)
  • Author(s): Kazuho Watanabe and Teemu Roos
  • Journal Title: Journal of Machine Learning Research Volume: to be assigned
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Journal Article] Vector quantization based on epsilon-insensitive mixture models (2015)
  • Author(s): Kazuho Watanabe
  • Journal Title: Neurocomputing Volume: to be assigned Pages: 32-37
  • DOI: 10.1016/j.neucom.2015.01.081
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Journal Article] An alternative view of variational Bayes and asymptotic approximations of free energy (2012)
  • Author(s): K.Watanabe
  • Journal Title: Machine Learning Volume: 86 Issue: 2 Pages: 273-293
  • DOI: 10.1007/s10994-011-5264-5
  • Peer Reviewed
• [Journal Article] Divergence measures and a general framework for local variational approximation (2011)
  • Author(s): K. Watanabe, M. Okada, K. Ikeda
  • Journal Title: Neural Networks Volume: 24(10) Issue: 10 Pages: 1102-1109
  • DOI: 10.1016/j.neunet.2011.06.004
  • Peer Reviewed
• [Journal Article] Approximate Bayesian Estimation of Varying Binomial Process (2011)
  • Author(s): K.Watanabe, M.Okada
  • Journal Title: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences Volume: E94-A Issue: 12 Pages: 2879-2885
  • DOI: 10.1587/transfun.E94.A.2879
  • NAID: 10030534322
  • ISSN: 0916-8508, 1745-1337
  • Peer Reviewed
• [Presentation] Biclustering Multivariate Data for Correlated Subspace Mining (2015)
  • Author(s): Kazuho Watanabe, Hsiang-Yun Wu, Yusuke Niibe, Shigeo Takahashi, and Issei Fujishiro
  • Organizer: IEEE Pacific Visualization Symposium 2015 (PacificVis 2015)
  • Place of Presentation: Hangzhou, China
  • Year and Date: 2015-04-14 – 2015-04-17
• [Presentation] レート歪み関数と測度の最適化 (2014)
  • Author(s): 渡辺一帆
  • Organizer: 第３７回情報理論とその応用シンポジウム（SITA2014）ワークショップ
  • Place of Presentation: 富山県黒部市
  • Year and Date: 2014-12-09 – 2014-12-12
  • Invited
• [Presentation] Analysis of variational Bayesian latent Dirichlet allocation: weaker sparsity than MAP (2014)
  • Author(s): Shinichi Nakajima, Issei Sato, Masashi Sugiyama, Kazuho Watanabe, and Hiroko Kobayashi
  • Organizer: Neural Information Processing Systems (NIPS2014)
  • Place of Presentation: Montreal, Canada
  • Year and Date: 2014-12-08 – 2014-12-13
• [Presentation] Asymptotic Analysis of Variational Bayesian Latent Dirichlet Allocation (2014)
  • Author(s): Shinichi Nakajima, Issei Sato, Masashi Sugiyama, Kazuho Watanabe, and Hiroko Kobayashi
  • Organizer: 情報論的学習理論ワークショップ (IBIS2014)
  • Place of Presentation: 名古屋大学
  • Year and Date: 2014-11-16 – 2014-12-19
• [Presentation] Rate-distortion analysis for an epsilon-insensitive loss function (2014)
  • Author(s): Kazuho Watanabe
  • Organizer: Workshop on Information Theoretic Methods in Science and Engineering (WITMSE2014)
  • Place of Presentation: Honolulu, USA
  • Year and Date: 2014-07-05 – 2014-07-07
  • Invited
• [Presentation] Bayesian properties of normalized maximum likelihood and its fast computation (2014)
  • Author(s): Andrew Barron, Teemu Roos, and Kazuho Watanabe
  • Organizer: IEEE International Symposium on Information Theory (ISIT2014)
  • Place of Presentation: Honolulu, USA
  • Year and Date: 2014-06-29 – 2014-07-04
• [Presentation] Bayesian Properties of Normalized Maximum Likelihood and its Fast Computation (2014)
  • Author(s): A. R. Barron, T. Roos, K. Watanabe
  • Organizer: 2014 IEEE International Symposium on Information Theory
  • Place of Presentation: Hawaii Convention Center, Honolulu, Hawaii, USA
• [Presentation] Rate-Distortion Function for Gamma Sources under Absolute-Log Distortion Measure (2013)
  • Author(s): K. Watanabe, S. Ikeda
  • Organizer: 2013 IEEE International Symposium on Information Theory
  • Place of Presentation: Istanbul Convention & Exhibition Center (ICEC) , Istanbul, Turkey
• [Presentation] Achievability of Asymptotic Minimax Optimality in Online and Batch Coding (2013)
  • Author(s): K. Watanabe, T. Roos, P. Myllymaki
  • Organizer: 2013 Workshop on Information Theoretic Methods in Science and Engineering
  • Place of Presentation: 東京大学山上会館，文京区，東京都
  • Invited
• [Presentation] Rate-Distortion Bounds for an Epsilon-Insensitive Distortion Measure (2013)
  • Author(s): K. Watanabe
  • Organizer: IEEE Information Theory Workshop
  • Place of Presentation: School of Engineering, University of Seville, Sevilla, Spain
• [Presentation] Vector Quantization Using Mixture of Epsilon-Insensitive Components (2013)
  • Author(s): K. Watanabe
  • Organizer: 2013International Conference on Neural Information Processing
  • Place of Presentation: EXCO (Daegu Exhibition & Convention Center), Daegu, Korea
• [Presentation] Achievability of Asymptotic Minimax Regret in Online and Batch Prediction (2013)
  • Author(s): K. Watanabe, T. Roos, P. Myllymaki
  • Organizer: Asian Conference on Machine Learning
  • Place of Presentation: Australian National University, Canberra, Australia
• [Presentation] Phase diagrams of a variational Bayesian approach with ARD prior in NIRS-DOT (2011)
  • Author(s): A.Miyamoto, K.Watanabe, K.Ikeda, M.Sato
  • Organizer: International Joint Conference on Neural Networks
  • Place of Presentation: San Jose, California, USA
• [Presentation] Minimum variational stochastic complexity and average generalization error in latent variable models (2011)
  • Author(s): K.Watanabe
  • Organizer: Workshop on Information Theoretic Methods in Science and Engineering(招待講演)
  • Place of Presentation: Helsinki, Finland
• [Presentation] Convex formulation for nonparametric estimation of mixing distribution
  • Author(s): K. Watanabe, S. Ikeda
  • Organizer: Workshop on Information Theoretic Methods in Science and Engineering
  • Place of Presentation: Amsterdam, Netherlands
  • Invited
• [Presentation] Packet Loss Rate Estimation with Active and Passive Measurements
  • Author(s): A. Miyamoto, K. Watanabe, K. Ikeda
  • Organizer: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference
  • Place of Presentation: Hollywood, California, USA
• [Presentation] 自由エネルギーによる潜在変数推定精度の計算法
  • Author(s): 山崎 啓介，渡辺一帆，梶 大介
  • Organizer: 情報論的学習理論ワークショップ
  • Place of Presentation: 東京都文京区
• [Presentation] Rate-distortion function for gamma sources under absolute-log distortion
  • Author(s): K. Watanabe, S. Ikeda
  • Organizer: 情報理論とその応用シンポジウム
  • Place of Presentation: 大分県別府市
• [Presentation] 漸近的ミニマックス性とデータ数の知識の関係について
  • Author(s): 渡辺一帆，テームルース，ペトリミリマキ
  • Organizer: 電子情報通信学会情報論的学習理論と機械学習研究会
  • Place of Presentation: 名古屋工業大学
• [Presentation] Rate-Distortion Function for Gamma Sources under Absolute-Log Distortion Measure
  • Author(s): K. Watanabe, S. Ikeda
  • Organizer: IEEE International Symposium on Information Theory
  • Place of Presentation: Istanbul, Turkey
• [Presentation] NIRS-DOT のための変分ベイズによる逆問題解法
  • Author(s): 宮本敦史，渡辺一帆，池田和司，佐藤雅昭
  • Organizer: 第５５回システム制御情報学会研究発表講演会
  • Place of Presentation: 大阪大学
• [Presentation] ブレグマン情報量を用いたベイズ学習の局所変分近似
  • Author(s): 渡辺一帆，岡田真人，池田和司
  • Organizer: 電子情報通信学会ニューロコンピューティング研究会
  • Place of Presentation: 神戸大学
• [Presentation] 付加的計測を用いたネットワークトモグラフィ
  • Author(s): 宮本敦史, 渡辺一帆, 池田和司
  • Organizer: 電子情報通信学会ソサイエティ大会
  • Place of Presentation: 北海道大学
• [Presentation] アクティブ計測とパッシブ計測を用いたネットワークトモグラフィ
  • Author(s): 宮本敦史, 渡辺一帆, 池田和司
  • Organizer: 電子情報通信学会情報論的学習理論と機械学習研究会
  • Place of Presentation: 奈良女子大学
• [Presentation] 混合分布のノンパラメトリック推定と最小ダイバージェンス法
  • Author(s): 渡辺一帆，池田思朗
  • Organizer: 電子情報通信学会情報論的学習理論と機械学習研究会
  • Place of Presentation: 奈良女子大学
• [Presentation] アクティブ計測とパッシブ計測を用いたパケットロス率推定法
  • Author(s): 宮本敦史, 渡辺一帆, 池田和司
  • Organizer: 電子情報通信学会情報論的学習理論と機械学習研究会
  • Place of Presentation: 統計数理研究所
• [Remarks] 代表者のホームページ
  • URL: http://www.lisl.cs.tut.ac.jp/wkazuho/index.html
• [Remarks] 研究代表者のホームページ
  • URL: http://hawaii.naist.jp/~wkazuho/index-j.html
• [Remarks]
  • URL: http://hawaii.naist.jp/~wkazuho/list-e.html