Developing Fast Machine Learning Algorithm based on Auxiliary Function-based Optimization Approach

2015 Fiscal Year Final Research Report

• Project/Area Number: 26540090
• Research Category: Grant-in-Aid for Challenging Exploratory Research
• Allocation Type: Multi-year Fund
• Research Field: Perceptual information processing
• Research Institution: National Institute of Informatics
• Principal Investigator: ONO Nobutaka 国立情報学研究所, 情報学プリンシプル研究系, 准教授 (80334259)
• Project Period (FY): 2014-04-01 – 2016-03-31
• Keywords: 深層学習 / 補助関数法 / ニューラルネットワーク / 最適化 / パターン認識

Outline of Final Research Achievements

We derived a quadratic auxiliary function with a look-up table for 2-layer neural network with a tangent hyperbolic activation function. Also, for training multi-layer neural network, we showed the auxiliary function could be designed from the output to the input recursively, and improved the algorithm based on a new concept of “back propagation of target”. By experiments with MIST handwritten digit database, we showed the derived algorithm needed much less iterations for convergence than a conventional adaptive gradient method.

Free Research Field

音響信号処理