Development and application of the finite state Markov chain approximation method for control systems described by stochastic differential equations

2022 Fiscal Year Final Research Report

• Project/Area Number: 20K11989
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61040:Soft computing-related
• Research Institution: Osaka University
• Principal Investigator: Suzuki Yasuyuki 大阪大学, 大学院基礎工学研究科, 講師 (30631874)
• Project Period (FY): 2020-04-01 – 2023-03-31
• Keywords: 確率微分方程式 / Fokker-Planck方程式 / 有限状態マルコフ鎖モデル / 強化学習 / データ同化

Outline of Final Research Achievements

In this study, we constructed a framework for approximating the Fokker-Planck equation by the finite element method to a finite state Markov chain model, and further developed a foundation for applying the obtained finite state Markov chain to reinforcement learning and data assimilation problems. As a concrete example, we performed a finite state Markov chain approximation of the intermittent control model of human standing posture, which is described by a nonlinear mechanical system in which the probability flow changes steeply depending on the state. We revealed that the method we developed in this study can accurately obtain numerical approximate solutions for initial value and boundary value problems of the system.

Free Research Field

ソフトコンピューティング

Academic Significance and Societal Importance of the Research Achievements

Fokker-Planck方程式の動態を近似する数値計算手法は，必ずしも十分に整備されていなかった．本研究でFokker-Planck方程式を有限要素解析に基づき，有限状態マルコフ鎖モデルによって精度よく近似する枠組みの構築したことにより，確率微分方程式で記述される様々なシステムのより詳細な解析への道筋が開けたことになる．さらに本研究では，有限状態マルコフ鎖を強化学習やデータ同化問題に適用する基盤整備を進めた．これにより，確率的に変動する生命現象を様々な手法により取り扱う枠組みの整備が進み，研究の更なる飛躍が期待される．