• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Stochastic Fixed Point Optimization Algorithm and Its Application to Ensemble Learning

Research Project

Project/Area Number 18K11184
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 60020:Mathematical informatics-related
Research InstitutionMeiji University

Principal Investigator

Iiduka Hideaki  明治大学, 理工学部, 専任教授 (50532280)

Project Period (FY) 2018-04-01 – 2021-03-31
Project Status Completed (Fiscal Year 2020)
Budget Amount *help
¥3,510,000 (Direct Cost: ¥2,700,000、Indirect Cost: ¥810,000)
Fiscal Year 2020: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2019: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2018: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Keywords確率的最適化 / 不動点 / アンサンブル学習 / 最適化アルゴリズム
Outline of Final Research Achievements

We consider a classifier ensemble problem with sparsity and diversity learning and show that the problem can be formulated as a stochastic optimization problem with fixed point constraint. For such a problem, we propose an algorithm referred to as the stochastic fixed point optimization algorithm and perform a convergence analysis for three types of learning rate: constant learning rate, decreasing learning rate, and a learning rate computed by line searches. In the case of a constant learning rate, the results indicate that a sufficiently small constant learning rate allows a solution to the problem to be approximated. In the case of a decreasing learning rate, conditions are shown under which the algorithm converges to a solution. For the third case, a variation of the proposed algorithm also achieves convergence to a solution. The high classification accuracies of the proposed algorithms are demonstrated through numerical comparisons with the conventional algorithm.

Academic Significance and Societal Importance of the Research Achievements

疎性や多様性を考慮したアンサンブル学習においては、大規模かつ複雑な確率的最適化問題を解く必要がある。しなしながら、従来のアンサンブル学習法は、その問題の大幅な緩和やその問題の解へ収束する保証がない学習アルゴリズムに基づいており、本来達成すべきアンサンブル学習法の性能を満たしていない。本研究での提案手法は、その問題に直接適用できる不動点最適化アルゴリズムに基づくアンサンブル学習法であり、世界的に例のない新解法である。本研究の成果は、従来アンサンブル学習法の適用範囲に関する改善に多大な貢献ができることから応用数学的観点のみならず、工学的観点から見ても意義があると言える。

Report

(4 results)
  • 2020 Annual Research Report   Final Research Report ( PDF )
  • 2019 Research-status Report
  • 2018 Research-status Report
  • Research Products

    (17 results)

All 2020 2019 2018 Other

All Journal Article (13 results) (of which Int'l Joint Research: 3 results,  Peer Reviewed: 13 results,  Open Access: 8 results) Presentation (3 results) (of which Int'l Joint Research: 2 results,  Invited: 3 results) Remarks (1 results)

  • [Journal Article] Fixed point quasiconvex subgradient method2020

    • Author(s)
      Kazuhiro Hishinuma, Hideaki Iiduka
    • Journal Title

      European Journal of Operational Research

      Volume: 282 Issue: 2 Pages: 428-437

    • DOI

      10.1016/j.ejor.2019.09.037

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Decentralized Hierarchical Constrained Convex Optimization2020

    • Author(s)
      Hideaki Iiduka
    • Journal Title

      Optimization and Engineering

      Volume: 21 Issue: 1 Pages: 181213-181213

    • DOI

      10.1007/s11081-019-09440-7

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Parallel computing proximal method for nonsmooth convex optimization with fixed point constraints of quasi-nonexpansive mappings2020

    • Author(s)
      Kengo Shimizu, Kazuhiro Hishinuma, Hideaki Iiduka
    • Journal Title

      Applied Set-Valued Analysis and Optimization

      Volume: 2 Issue: 1 Pages: 1-17

    • DOI

      10.23952/asvao.2.2020.1.01

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Efficiency of Inexact Fixed Point Quasiconvex Subgradient Method2020

    • Author(s)
      Kazuhiro Hishinuma, Hideaki Iiduka
    • Journal Title

      Linear and Nonlinear Analysis

      Volume: 6 Pages: 3548-3548

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Computation Time of Iterative Methods for Nonsmooth Convex Optimization With Fixed Point Constraints of Quasi-Nonexpansive Mappings2020

    • Author(s)
      Kengo Shimizu, Hideaki Iiduka
    • Journal Title

      Linear and Nonlinear Analysis

      Volume: 6 Pages: 281286-281286

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Training Deep Neural Networks Using Conjugate Gradient-like Methods2020

    • Author(s)
      Hideaki Iiduka, Yu Kobayashi
    • Journal Title

      Electronics

      Volume: 9 Issue: 11 Pages: 1809-1809

    • DOI

      10.3390/electronics9111809

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Hybrid Riemannian Conjugate Gradient Methods with Global Convergence Properties2020

    • Author(s)
      Hiroyuki Sakai, Hideaki Iiduka
    • Journal Title

      Computational Optimization and Applications

      Volume: 77 Issue: 3 Pages: 811830-811830

    • DOI

      10.1007/s10589-020-00224-9

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Stochastic Fixed Point Optimization Algorithm for Classifier Ensemble2020

    • Author(s)
      Hideaki Iiduka
    • Journal Title

      IEEE Transactions on Cybernetics

      Volume: - Issue: 10 Pages: 4370-4380

    • DOI

      10.1109/tcyb.2019.2921369

    • Related Report
      2019 Research-status Report
    • Peer Reviewed
  • [Journal Article] Incremental Proximal Method for Nonsmooth Convex Optimization With Fixed Point Constraints of Quasi-nonexpansive Mappings2019

    • Author(s)
      Haruhi Oishi, Yu Kobayashi, Hideaki Iiduka
    • Journal Title

      Linear and Nonlinear Analysis

      Volume: 5 Pages: 477493-477493

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Convergence Analysis of Incremental and Parallel Line Search Subgradient Methods in Hilbert Space2019

    • Author(s)
      Kazuhiro Hishinuma, Hideaki Iiduka
    • Journal Title

      Journal of Nonlinear and Convex Analysis

      Volume: 20 Pages: 19371947-19371947

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Incremental and Parallel Machine Learning Algorithms With Automated Learning Rate Adjustments2019

    • Author(s)
      Kazuhiro Hishinuma, Hideaki Iiduka
    • Journal Title

      Frontiers in Robotics and AI

      Volume: 6 Pages: 77-77

    • DOI

      10.3389/frobt.2019.00077

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Iterative Methods for Parallel Convex Optimization With Fixed Point Constraints2019

    • Author(s)
      Kaito Sakurai, Takayuki Jimba, Hideaki Iiduka
    • Journal Title

      Journal of Nonlinear and Variational Analysis

      Volume: 3 Issue: 2 Pages: 115126-115126

    • DOI

      10.23952/jnva.3.2019.2.01

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Distributed Optimization for Network Resource Allocation with Nonsmooth Utility Functions2018

    • Author(s)
      Hideaki Iiduka
    • Journal Title

      IEEE Transactions on Control of Network Systems

      Volume: 印刷中 Issue: 4 Pages: 1354-1365

    • DOI

      10.1109/tcns.2018.2889011

    • Related Report
      2018 Research-status Report
    • Peer Reviewed
  • [Presentation] 複雑制約付き凸最適化とその応用 -不動点理論で端緒を開く-2019

    • Author(s)
      飯塚 秀明
    • Organizer
      日本オペレーションズ・リサーチ学会 2019年秋季研究発表会
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] Fixed Point Algorithms and Their Applications2019

    • Author(s)
      Hideaki Iiduka
    • Organizer
      The International Conference on Nonlinear Analysis and Convex Analysis-International Conference on Optimization: Techniques and Applications
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] Decentralized Optimization and Its Applications2018

    • Author(s)
      Hideaki Iiduka
    • Organizer
      The 6th Asian Conference on Nonlinear Analysis and Optimization
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research / Invited
  • [Remarks] 数理最適化研究室へようこそ

    • URL

      https://iiduka.net/

    • Related Report
      2020 Annual Research Report 2019 Research-status Report 2018 Research-status Report

URL: 

Published: 2018-04-23   Modified: 2022-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi