Stochastic Fixed Point Optimization Algorithm and Its Application to Ensemble Learning

2020 Fiscal Year Final Research Report

• Project/Area Number: 18K11184
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 60020:Mathematical informatics-related
• Research Institution: Meiji University
• Principal Investigator: Iiduka Hideaki 明治大学, 理工学部, 専任教授 (50532280)
• Project Period (FY): 2018-04-01 – 2021-03-31
• Keywords: 確率的最適化 / 不動点 / アンサンブル学習 / 最適化アルゴリズム

Outline of Final Research Achievements

We consider a classifier ensemble problem with sparsity and diversity learning and show that the problem can be formulated as a stochastic optimization problem with fixed point constraint. For such a problem, we propose an algorithm referred to as the stochastic fixed point optimization algorithm and perform a convergence analysis for three types of learning rate: constant learning rate, decreasing learning rate, and a learning rate computed by line searches. In the case of a constant learning rate, the results indicate that a sufficiently small constant learning rate allows a solution to the problem to be approximated. In the case of a decreasing learning rate, conditions are shown under which the algorithm converges to a solution. For the third case, a variation of the proposed algorithm also achieves convergence to a solution. The high classification accuracies of the proposed algorithms are demonstrated through numerical comparisons with the conventional algorithm.

Free Research Field

最適化

Academic Significance and Societal Importance of the Research Achievements

疎性や多様性を考慮したアンサンブル学習においては、大規模かつ複雑な確率的最適化問題を解く必要がある。しなしながら、従来のアンサンブル学習法は、その問題の大幅な緩和やその問題の解へ収束する保証がない学習アルゴリズムに基づいており、本来達成すべきアンサンブル学習法の性能を満たしていない。本研究での提案手法は、その問題に直接適用できる不動点最適化アルゴリズムに基づくアンサンブル学習法であり、世界的に例のない新解法である。本研究の成果は、従来アンサンブル学習法の適用範囲に関する改善に多大な貢献ができることから応用数学的観点のみならず、工学的観点から見ても意義があると言える。