Development of a general purpose game AI that improves humanly

2022 Fiscal Year Final Research Report

• Project/Area Number: 17K00514
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Entertainment and game informatics 1
• Research Institution: Matsue National College of Technology
• Principal Investigator: Hashimoto Tsuyoshi 松江工業高等専門学校, 情報工学科, 教授 (40420335)
• Project Period (FY): 2017-04-01 – 2023-03-31
• Keywords: 強化学習 / 反復広化法 / 顕著性マップ / 汎用ゲームAI / 生物学的制約導入 / 弾幕シューティングAI / Muzero / ガイスター

Outline of Final Research Achievements

Aiming to realize a general-purpose game AI that improves in a human-like manner, we developed methods on a variety of subjects. We proposed an iterative broadening method inspired by human mastery, and confirmed its effectiveness first on Tetris and later on reinforcement learning of Puyo-Puyo and Pomberman. We also focused on human gazing and saliency calculated from features such as color and edges of images, and proposed a reinforcement learning method using saliency maps, and confirmed its effectiveness in bullet-shooting AI and Ms. Pacman's reinforcement learning. Other achievements include Super Mario infinite 1-up learning, creation of human-like features by introducing biological constraints to music game reinforcement learning, development of a method for searching for purple pieces in Geister, and a method for learning with Muzero by creating a game-like environment similar to a real environment.

Free Research Field

ゲーム情報学

Academic Significance and Societal Importance of the Research Achievements

近年、AI強化学習の研究が世界中で盛んに行われているが、効率よく学習をさせるために、人間の学習に着目し、人間らしい強化学習を可能にするための手法をいくつか提案し、評価しやすいゲームAIで実装し実験によりその効果を示した。これら手法が強化学習に取り入れられることで、より少ない計算資源で高い学習効果を上げられる。また、音楽ゲーム強化学習への生物学的制約導入による人間らしさ創出、実環境に近い環境をゲーム化し学習する手法など、実社会に近い題材でも応用可能な成果を上げ強化学習の新たな方向性を示すことができた。