Parallel-Distributed Machine Learning Environment for Reinforcement Learning

2022 Fiscal Year Final Research Report

• Project/Area Number: 19K11994
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 60090:High performance computing-related
• Research Institution: National Institute of Advanced Industrial Science and Technology
• Principal Investigator: NAKADA HIDEMOTO 国立研究開発法人産業技術総合研究所, 情報・人間工学領域, 主任研究員 (80357631)
• Project Period (FY): 2019-04-01 – 2023-03-31
• Keywords: 分散計算 / 並列計算 / 強化学習

Outline of Final Research Achievements

In order to construct a framework for reinforcement learning in a parallel and distributed environment, we constructed a parallel and distributed system targeting a fast and highly functional language. We focused on the Julia language, which is rapidly expanding its use in machine learning, as a target language and implemented an actor mechanism suitable for implementing a reinforcement learning framework. Furthermore, considering the use on large-scale computer systems with fast networks, which are often used for large-scale machine learning, we optimized our framewrk for the fast network library and confirmed a significant performance improvement.

Free Research Field

分散並列計算

Academic Significance and Societal Importance of the Research Achievements

本研究では、並列分散強化学習をターゲットとして研究を進めたが、高性能計算機環境に適した並列分散実行環境の応用範囲は強化学習への適用にとどまらず、広く適用可能である。特にPythonを置き換える可能性のある言語として最近急速に注目を集めているJulia言語で実現したことには大きな社会的意義があると考える。また直感的に利用可能なActorをAPIとして採用することで、多くのユーザや応用分野に資することができると考える。