• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development and application of the finite state Markov chain approximation method for control systems described by stochastic differential equations

Research Project

Project/Area Number 20K11989
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 61040:Soft computing-related
Research InstitutionOsaka University

Principal Investigator

Suzuki Yasuyuki  大阪大学, 大学院基礎工学研究科, 講師 (30631874)

Project Period (FY) 2020-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2022: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2021: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2020: ¥1,950,000 (Direct Cost: ¥1,500,000、Indirect Cost: ¥450,000)
Keywords確率微分方程式 / Fokker-Planck方程式 / 有限状態マルコフ鎖モデル / 強化学習 / データ同化 / 有限要素解析 / 有限状態マルコフ / 有限状態マルコフ鎖近似 / ハイブリッド力学系
Outline of Research at the Start

連続時間の確率的なシステムの動態は確率微分方程式で記述される.確率微分方程式で記述される制御系の動態解析には,系のFokker-Planck方程式の解として得られる状態点の確率密度関数の時間発展や定常分布を解析することが有用である.しかしながら,Fokker-Planck方程式の動態を近似する数値計算手法は,必ずしも十分整備されていない.本研究では,確率微分方程式で記述される制御系の初期値問題・境界値問題を,Fokker-Planck方程式の有限要素解析に基づき,有限状態マルコフ鎖モデルによって精度よく近似する方法を開発する.さらにこれを強化学習やデータ同化問題に適用し,有用性を明らかにする.

Outline of Final Research Achievements

In this study, we constructed a framework for approximating the Fokker-Planck equation by the finite element method to a finite state Markov chain model, and further developed a foundation for applying the obtained finite state Markov chain to reinforcement learning and data assimilation problems. As a concrete example, we performed a finite state Markov chain approximation of the intermittent control model of human standing posture, which is described by a nonlinear mechanical system in which the probability flow changes steeply depending on the state. We revealed that the method we developed in this study can accurately obtain numerical approximate solutions for initial value and boundary value problems of the system.

Academic Significance and Societal Importance of the Research Achievements

Fokker-Planck方程式の動態を近似する数値計算手法は,必ずしも十分に整備されていなかった.本研究でFokker-Planck方程式を有限要素解析に基づき,有限状態マルコフ鎖モデルによって精度よく近似する枠組みの構築したことにより,確率微分方程式で記述される様々なシステムのより詳細な解析への道筋が開けたことになる.さらに本研究では,有限状態マルコフ鎖を強化学習やデータ同化問題に適用する基盤整備を進めた.これにより,確率的に変動する生命現象を様々な手法により取り扱う枠組みの整備が進み,研究の更なる飛躍が期待される.

Report

(5 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Research-status Report
  • 2020 Research-status Report
  • Products Report
  • Research Products

    (4 results)

All 2023 2021 2020 Other

All Int'l Joint Research (1 results) Journal Article (3 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 2 results,  Open Access: 1 results)

  • [Int'l Joint Research] Istituto Italiano di Tecnologia(イタリア)

    • Related Report
      2020 Research-status Report
  • [Journal Article] A Markov chain approximation of switched Fokker?Planck equations for a model of on?off intermittency in the postural control during quiet standing2023

    • Author(s)
      Suzuki Yasuyuki、Togame Keigo、Nakamura Akihiro、Nomura Taishin
    • Journal Title

      Communications in Nonlinear Science and Numerical Simulation

      Volume: 126 Pages: 107488-107488

    • DOI

      10.1016/j.cnsns.2023.107488

    • Related Report
      Products Report
    • Peer Reviewed
  • [Journal Article] 確率的ハイブリッド力学系の有限要素解析に基づくヒト立位姿勢間欠制御モデルの有限状態マルコフ鎖近似2021

    • Author(s)
      鈴木康之,十亀敬伍,中村晃大,野村泰伸
    • Journal Title

      電子情報通信学会技術研究報告書-IEICE Technical Report-

      Volume: 121 Pages: 108-113

    • Related Report
      2021 Research-status Report
  • [Journal Article] Postural instability via a loss of intermittent control in elderly and patients with Parkinson’s disease: A model-based and data-driven approach2020

    • Author(s)
      Suzuki Yasuyuki、Nakamura Akihiro、Milosevic Matija、Nomura Kunihiko、Tanahashi Takao、Endo Takuyuki、Sakoda Saburo、Morasso Pietro、Nomura Taishin
    • Journal Title

      Chaos: An Interdisciplinary Journal of Nonlinear Science

      Volume: 30 Issue: 11 Pages: 113140-113140

    • DOI

      10.1063/5.0022319

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research

URL: 

Published: 2020-04-28   Modified: 2025-03-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi