Learning of a sparse code by a constrained optimization

• Project/Area Number: 19700219
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Single-year Grants
• Research Field: Sensitivity informatics/Soft computing
• Research Institution: Kyoto University (2008-2009); Nara Institute of Science and Technology (2007)
• Principal Investigator: SHIN-ICHI Maeda Kyoto University, 情報学研究科, 助教 (20379530)
• Project Period (FY): 2007 – 2009
• Project Status: Completed (Fiscal Year 2009)
• Budget Amount: ¥2,600,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥300,000); Fiscal Year 2009: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000); Fiscal Year 2008: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000); Fiscal Year 2007: ¥1,300,000 (Direct Cost: ¥1,300,000)
• Keywords: 階層ニューラルネットワーク / Contrastive Divergence Learning / 制約付き最適化 / EMアルゴリズム / スパースコーディング / 階層型モデル / 特徴抽出 / contrastive divergence / ニューラルネットワーク / ボルツマンマシン / autoencoder / pretraining / 自由エネルギー最小化 / companding function

Research Abstract

Hinton et al. (2006) showed that the performance of hierarchical neural networks couldbe greatly improved by an innovation of the learning algorithm and training with a largeamount of data. In this research project, I aim to clear up the causes which enable anefficient learning. As the results, I've succeeded to clearly show the reason why thelearning by EM algorithm freezes in some cases, generalize contrastive divergencelearning which was used in the training of the hierarchical neural networks and newlyderive the convergence condition of the algorithm. We also successfully demonstrated thatan efficient coding of natural images can be obtained by the learning based on aconstrained optimization.

Research Products

• [Journal Article] Sparse Bayesian Learning of Filters for Efficient Image Expansion (2010)
  • Author(s): A. Kanemura, S. Maeda, S. Ishii.
  • Journal Title: IEEE Transactions on image processing 19(6) Pages: 1480-1490
  • NAID: 120002381088
  • Peer Reviewed
• [Journal Article] Learning a multidimensional companding function for lossy source coding (2009)
  • Author(s): S. Maeda, S. Ishii.
  • Journal Title: Neural Networks 22(7) Pages: 998-1010
  • Peer Reviewed
• [Journal Article] Learning a multidimensional companding function for lossy source coding (2009)
  • Author(s): S.Maeda, S.Ishii
  • Journal Title: Neural Networks 22(7) Pages: 998-1010
  • Peer Reviewed
• [Presentation] オンライン適応自然勾配法による囲碁の盤面評価の学習 (2010)
  • Author(s): 富澤弘貴, 前田新一, 石井信
  • Organizer: 電子情報通信学会NC研究会
  • Place of Presentation: 玉川
  • Year and Date: 2010-03-11
• [Presentation] オンライン適応自然勾配法による囲碁の盤面評価の学習 (2010)
  • Author(s): 富澤弘貴, 前田新一, 石井信
  • Organizer: 電子情報通信学会 NC研究会
  • Place of Presentation: 玉川大学
  • Year and Date: 2010-03-11
• [Presentation] Learning of Go board state evaluation function by artificial neural network (2009)
  • Author(s): H. Tomizawa, S. Maeda, S. Ishii
  • Organizer: International Conference on Neural Information Processing
  • Place of Presentation: バンコク
  • Year and Date: 2009-12-03
• [Presentation] Learning of Go board state evaluation function by artificial neural network (2009)
  • Author(s): H.Tomizawa, S.Maeda, S.Ishii
  • Organizer: International Conference on Neural Information Processing
  • Place of Presentation: タイ,バンコク
  • Year and Date: 2009-12-03
• [Presentation] Detailed Balance Learningによるマルコフ連鎖の学習 (2009)
  • Author(s): 前田新一, 青木佑紀, 石井信
  • Organizer: 日本神経回路学会第19回全国大会
  • Place of Presentation: 仙台
  • Year and Date: 2009-09-25
• [Presentation] Detailed Balance Learningによるマルコフ連鎖の学習 (2009)
  • Author(s): 前田新一, 青木佑紀, 石井信
  • Organizer: 日本神経回路学会 第19回全国大会
  • Place of Presentation: 東北大学
  • Year and Date: 2009-09-25
• [Presentation] ニューラルネットワークによる囲碁の盤面評価の学習 (2009)
  • Author(s): 富澤弘貴, 前田新一, 石井信
  • Organizer: 日本神経回路学会第19回全国大会
  • Place of Presentation: 仙台
  • Year and Date: 2009-09-24
• [Presentation] 平均場近似を用いたHelmholtz Machineの学習 (2009)
  • Author(s): 青木佑紀, 前田新一, 石井信
  • Organizer: 電子情報通信学会NC研究会
  • Place of Presentation: 玉川
  • Year and Date: 2009-03-11
• [Presentation] 平均場近似を用いたHelmholtz Machineの学習 (2009)
  • Author(s): 青木佑紀, 前田新一, 石井信
  • Organizer: 電子情報通信学会NC研究会
  • Place of Presentation: 玉川大学
  • Year and Date: 2009-03-11
• [Presentation] スパースベイズ学習による画像拡大フィルタの獲得 (2008)
  • Author(s): 兼村厚範, 前田新一, 石井信
  • Organizer: 第23回信号処理シンポジウム
  • Place of Presentation: 金沢
  • Year and Date: 2008-11-13
• [Presentation] 疎ベイズ解像度合成 (2008)
  • Author(s): 兼村厚範, 前田新一, 石井信
  • Organizer: 日本神経回路学会第18回全国大会
  • Place of Presentation: つくば
  • Year and Date: 2008-09-25
• [Presentation] Subspace selection for resolution synthesis (2008)
  • Author(s): A. Kanemura, S. Maeda, S. Ishii
  • Organizer: 部分空間法研究会2008
  • Place of Presentation: 軽井沢
  • Year and Date: 2008-07-28
• [Presentation] Optimization of parametric companding function for an efficient coding (2007)
  • Author(s): Shin-ichi Maeda, Shin Ishii
  • Organizer: IEEE International Symposium on Signal Processing and Information Technology
  • Place of Presentation: エジプト カイロ
  • Year and Date: 2007-12-16
• [Presentation] Optimization of parametric companding function for an efficient coding (2007)
  • Author(s): S. Maeda, S. Ishii
  • Organizer: International Conference on Neural Information Processing
  • Place of Presentation: 北九州
  • Year and Date: 2007-11-14
• [Presentation] Convergence analysis of the EM algorithm and Joint minimization of free energy (2007)
  • Author(s): S. Maeda, S. Ishii
  • Organizer: IEEE International Workshop on Machine Learning for Signal Processing
  • Place of Presentation: テッサロニキ
  • Year and Date: 2007-08-28
• [Presentation] Convergence analysis of the EM algorithm and joint minimization of free energy (2007)
  • Author(s): Shin-ichi Maeda, Shin Ishii
  • Organizer: IEEE International Workshop on Machine Learning for Signal Processing
  • Place of Presentation: ギリシャ テッサロニキ
  • Year and Date: 2007-08-28
• [Remarks]
  • URL: http://hawaii.sys.i.kyoto-u.ac.jp/~ichi/publication.htm