From "Exploration" To "Thinking" - Development of Chaos Dynamics through Reinforcement Learning

2019 Fiscal Year Final Research Report

• Project/Area Number: 15K00360
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Intelligent robotics
• Research Institution: Oita University
• Principal Investigator: SHIBATA Katsunari 大分大学, 理工学部, 教授 (10260522)
• Project Period (FY): 2015-04-01 – 2020-03-31
• Keywords: ダイナミック強化学習 / 感度 / 感度調整学習 / カオスニューラルネット / 思考 / 探索 / ダイナミクス / 汎用人工知能

Outline of Final Research Achievements

I could not reach the initial goal that is to establish the algorithm of reinforcement learning using a chaos neural network (NN), which I have proposed, and then the emergence of “primitive thinking” on the basis of the hypothesis that “exploration” grows into “thinking” through reinforcement learning.
On the other hand, I have proposed an index “sensitivity” in each neuron to control the chaoticity of the network globally, and also “sensitivity adjustment learning” to learn it. It can be used as an index for generating chaos, and it can also be used to solve the vanishing/exploding gradient problem in gradient-based learning. Furthermore, completely new reinforcement learning named “Dynamic Reinforcement Learning” in which the present output value is not learned directly but dynamics is learned by adjusting the sensitivity according to TD error (the difference of actual state value from its prediction), has come up.

Free Research Field

人工知能

Academic Significance and Societal Importance of the Research Achievements

各ニューロンのローカルな指標「感度」でニューラルネット全体のダイナミクスを制御すること，さらに，従来の静的な発想に基づく「現在の出力値を目的のものに近づける」ための学習という考えから脱し，動的な処理の学習に向けた「評価が良い場合は再現性を上げるためダイナミクスを収束へ，悪い場合は探索を強化するためダイナミクスを発散(カオス)へ」という「ダイナミック強化学習」は，全く新しい学習パラダイムを切り拓くものである。今後，より高次な機能が求められるに従い，求められるものが静的なものからより動的なものへとシフトしていく中で，学習の新たな根本原理としての役割を担うポテンシャルを持っていると期待している。