• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Autonomous, Harmonious and Purposive Acquisition of Various Functions of Robots by Reinforcement Learning and the Relation to the Intelligence Formation

Research Project

Project/Area Number 15300064
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Perception information processing/Intelligent robotics
Research InstitutionOita University

Principal Investigator

SHIBATA Katsunari  Oita University, Faculty of Engineering, Associate Professor, 工学部, 助教授 (10260522)

Project Period (FY) 2003 – 2006
Project Status Completed (Fiscal Year 2006)
Budget Amount *help
¥6,500,000 (Direct Cost: ¥6,500,000)
Fiscal Year 2006: ¥1,300,000 (Direct Cost: ¥1,300,000)
Fiscal Year 2005: ¥1,600,000 (Direct Cost: ¥1,600,000)
Fiscal Year 2004: ¥2,100,000 (Direct Cost: ¥2,100,000)
Fiscal Year 2003: ¥1,500,000 (Direct Cost: ¥1,500,000)
KeywordsReinforcement Learning / Recurrent Neural Network / Symbol / Robot / Spatial Abstraction / Intelligent Exploration / Temporal Abstraction / Emergence of Intelligence / 乗算ニューロン / 抽象化 / 予測 / 文脈 / 決定論的探索 / 一様探索 / ゲートニューロン / 報酬期待ニューロン / 実用的リカレント学習(PRL) / ニューラルネット / 知能形成 / コミュニケーション / シンボルグラウンディング問題 / カラー情報処理 / 成長型ニューラルネット / 隠れニューロン
Research Abstract

This research was aimed to show that by the learning using the training signals that are derived by reinforcement learning, various functions emerge according to the necessity in a neural network to which sensor signals are directly entered and whose outputs are motor commands. The main fruits are as follows.
1. It is said that neural networks are not good at symbol processing. However, it was shown that the output representation of a neural network became binary only by reinforcement learning.
2. It was shown that a real robot could learn box-pushing behavior using neural network without giving any informatio a about image processing, image recognition, or the given task.
3. It was shown that a real robot could learn to reach an object in some degree even in a quasi-real world where various objects and colorful leaflets exist.
4. It was shown that a recurrent neural network trained by reinforcement learning could learn some tasks that are thought to be relevant to the spatial or temporal abstraction.

Report

(5 results)
  • 2006 Annual Research Report   Final Research Report Summary
  • 2005 Annual Research Report
  • 2004 Annual Research Report
  • 2003 Annual Research Report
  • Research Products

    (55 results)

All 2007 2006 2005 2004 2003 Other

All Journal Article (48 results) Book (1 results) Publications (6 results)

  • [Journal Article] 階層型ニューラルネットにおける中間層ての適応的空間再構成と中間層レベルの汎化に基づく知識の継承2007

    • Author(s)
      柴田克成, 伊藤宏司
    • Journal Title

      計測自動制御学会論文集 Vol. 43, No.1

      Pages: 54-63

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 階層型ニューラルネットにおける中間層での適応的空間再構成と中間層レベルの汎化に基づく知識の継承2007

    • Author(s)
      柴田克成, 伊藤宏司
    • Journal Title

      計測自動制御学会論文集 Vol.43, No.1

      Pages: 54-63

    • NAID

      10018479914

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Acquisition of Deterministic Exploration Behavior by Reinforcement Learning2006

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Proc. of the 11th Int'l Symp. on Artificial Life and Robotics (CD-ROM)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] A Model to Explain the Emergence of Reward Expectancy neurons using Reinforcement Learning and Neural Network2006

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Neurocomputing Vol. 69

      Pages: 1327-1331

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Spatial Abstraction and Knowledge Transfer in Reinforcement Learning Using a Multi-Layer Neural Network2006

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Proc. of ICDL5 (Fifth Int'l Conf. on Development and Learning) (CD-ROM)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Annual Research Report 2006 Final Research Report Summary
  • [Journal Article] Learning of Deterministic Exploration and Temporal Abstraction in Reinforcement Learning2006

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Proc. of SICE-ICCAS (SICE-ICASE Int'l Joint Conf.) (CD-ROM)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Annual Research Report 2006 Final Research Report Summary
  • [Journal Article] Acquisition of Deterministic Exploration Behavior by Reinforcement Learning2006

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Proc. of the 11th Int'l Symp. on Artificial Life and Robotics (AROB) (CD-ROM)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] A Model to Explain the Emergence of Reward Expectancy neurons using Reinforcement Learning and Neural Network2006

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Neurocomputing Vol.69

      Pages: 1327-1331

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Acquisition of Deterministic Exploration Behavior by Reinforcement Learning2006

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Proc.of the 11th Int'l Symp.on Artificial Life and Robotics CD-ROM

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Effect of action selection on emergence of one-way communication using Q-learning2005

    • Author(s)
      Masanobu Nakanishi, Katsunari Shibata
    • Journal Title

      Proc. of 10^<th> Int'l Symp. On Artificial Life and Robotics (AROB)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] A model of emergence of reward expectancy neurons by reinforcement learning2005

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Proc. of 10^<th> Int'l Symp. On Artificial Life and Robotics (AROB)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Discretization of Series of Communication Signals in Noisy Environment by Reinforcement Learning2005

    • Author(s)
      Katsunari
    • Journal Title

      Proc. of the 7th Int'l Conf. in Adaptive and Natural Comniiting Algorithms

      Pages: 486-489

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] An Explanation of Emergence of Reward Expectancy Neurons Usine Reinforcement Learning and Neural Net2005

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Abstract Book of Fourteenth Annual Computational Neuroscience Meeting

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 強化学習による探索行動の学習2005

    • Author(s)
      柴田克成
    • Journal Title

      計測自動制御学会システム・情報部門学術講演会2005講演論文集

      Pages: 11-16

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary 2005 Annual Research Report
  • [Journal Article] A Model of Emergence of Reward Expectancy neurons by Reinforcement Learning2005

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Proc. of the 10th Int'l Symp. on Artificial Life and Robotics (AROB) (CD-ROM)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Effect of Action Selection on Emergence of One-way Communication Using Q-]earning2005

    • Author(s)
      Masanobu Nakanishi, Katsunari Shibata
    • Journal Title

      Proc. of the 10th Int'l Symp. on Artificial Life and Robotics (AROB). (CD-ROM)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Discretization of Series of Communication Signals in Noisy Environment by Reinforcement Learning2005

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Adaptive and Natural Computing Algorithms (Ribeiro et al. (eds.))

      Pages: 486-489

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] An Explanation of Emergence of Reward Expectancy Neurons Using Reinforcement Learning and Neural Net2005

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Abstract Book of Fourteenth Annual Computational Neuroscience Meeting

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Discretization of Series of Communication Signals in Noisy Environment by Reinforcement Learning2005

    • Author(s)
      Katsunari Shibata
    • Journal Title

      Proc.of the 7th Int'l Conf.in Adaptive and Natural Computing Algorithms

      Pages: 486-489

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Effect of action selection on emergence of one-way communication using Q-learning2005

    • Author(s)
      Masanobu Nakanishi, Katsunari Shibata
    • Journal Title

      Proc.of 10^<th> Int'l Symp.On Artificial Life and Robotics (AROB) CD-ROM

    • Related Report
      2004 Annual Research Report
  • [Journal Article] A model of emergence of reward expectancy neurons by reinforcement learning2005

    • Author(s)
      Shinya Ishii, Munetaka Shidara, Katsunari Shibata
    • Journal Title

      Proc.of 10^<th> Int'l Symp.On Artificial Life and Robotics (AROB) CD-ROM

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Discretization of Analog Communication Signals by Noise Addition in Communication Learning2004

    • Author(s)
      K.Shibata, M.Nakanishi
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 2

      Pages: 351-354

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Growing Neural Network with Hidden Neurons2004

    • Author(s)
      R.Kurinoz, M.Sugisaka, K.Shibata
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 1

      Pages: 144-147

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Learning of Reaching a Colored Object Based on Direct-Vision-Based Reinforcement Learning and Acquired Internal Representation2004

    • Author(s)
      K.Yuki, M.Sugisaka, K.Shibata
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 2

      Pages: 486-489

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Occurrence of state Confusion in the Learning of Communication Using Q-leaning2004

    • Author(s)
      M.Nakanishi, M.Sugisaka, K.Shibata
    • Journal Title

      Proc. of The 9th AROB (Infl Sympo. on Artificial Life and Robotics) Vol.2

      Pages: 663-666

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] コミュニケーションの強化学習におけるノイズ付加による連続値信号の離散化2004

    • Author(s)
      柴田克成
    • Journal Title

      電子情報通信学会技術研究報告 Vol. 103, No. 734

      Pages: 55-60

    • NAID

      110003232588

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 隠れニューロンの分離を伴う成長型ニューラルネットワーク2004

    • Author(s)
      栗野竜輔, 柴田克成
    • Journal Title

      電子情報通信学会技術研究報告 Vol. 103, No. 734

      Pages: 109-114

    • NAID

      110003232597

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Dynamics of a Recurrent Neural Network Acquired through the Learning of a Context-based Attention Task2004

    • Author(s)
      Katsunari Shibata, Masanori Sugisaka
    • Journal Title

      Artificial Life and Robotics Vol. 7

      Pages: 145-150

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 視覚センサ付き実ロボットによる箱押し行動の獲得 -強化学習によるセンサーモータ間トータル機能獲得への第一歩-2004

    • Author(s)
      柴田克成, 飯田大
    • Journal Title

      第14回インテリジェント・システム・シンポジウム(FAN Symposium)講演論文集

      Pages: 123-128

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary 2004 Annual Research Report
  • [Journal Article] 報酬期待ニューロンの強化学習による発現モデル2004

    • Author(s)
      石井慎也, 設楽宗孝, 柴田克成
    • Journal Title

      計測自動制御学会システム・情報部門学術講演会2004講演論文集

      Pages: 63-68

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Q学習に基づく一方向コミュニケーションの創発における行動選択の影響2004

    • Author(s)
      仲西賢展, 柴田克成
    • Journal Title

      計測自動制御学会システム・情報部門学術講演会2004講演論文集

      Pages: 157-162

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 多入力ニューラルネットの学習過程における中間層ニューロン数の影響2004

    • Author(s)
      藤田剛, 柴田克成
    • Journal Title

      第23回計測自動制御学会九州支部学術講演会予稿集

      Pages: 369-372

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] リカレントネットを用いた強化学習によるロボットの記憶に基づいた行動の学習2004

    • Author(s)
      幸和芳, 柴田克成
    • Journal Title

      第23回計測自動制御学会九州支部学術講演会予稿集

      Pages: 37-40

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 強化学習とニューラルネットを用いた報酬期待ニューロンの発現モデル2004

    • Author(s)
      石井慎也, 設楽宗孝, 柴田克成
    • Journal Title

      第23回計測自動制御学会九州支部学術講演会予稿集

      Pages: 305-308

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Occurrence of State Confusion in the Learning of Communication Using Q-leaning2004

    • Author(s)
      Masanobu Nakanishi, Masanori Sugisaka, Katsunari Shibata
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 2

      Pages: 663-666

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Learning of Reaching a Colored Object Based on Direct-Vision-Based Reinforcement Learning and Acquired Internal Representation2004

    • Author(s)
      Kazuyoshi Yuki, Masanori Sugisaka, Katsunari Shibata
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 2

      Pages: 486-489

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Growing Neural Network with Hidden Neurons2004

    • Author(s)
      Ryusuke Kurino, Masanori Sugisaka, Katsunari Shibata
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 1

      Pages: 144-147

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Discretization of Analog Communication Signals by Noise Addition in Communication Learning2004

    • Author(s)
      Katsunari Shibata, Masanobu Nakanishi
    • Journal Title

      Proc. of The 9th AROB (Int'l Sympo. on Artificial Life and Robotics) Vol. 2

      Pages: 351-354

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Dynamics of a Recurrent Neural Network Acquired through the Learning of a Context based Attention Task2004

    • Author(s)
      Katsunari Shibata, Masanori Sugisaka
    • Journal Title

      Artificial Life and Robotics Vol.7

      Pages: 145-150

    • Related Report
      2004 Annual Research Report
  • [Journal Article] 多入力ニューラルネットの学習過程における中間層ニューロン数の影響2004

    • Author(s)
      幸和芳, 柴田克成
    • Journal Title

      第23回計測自動制御学会九州支部学術講演会予稿集

      Pages: 369-372

    • Related Report
      2004 Annual Research Report
  • [Journal Article] Growing Neural Network for Acquisition of 2-layer structure2003

    • Author(s)
      R.Kurino, M.Sugisaka, K.Shibata
    • Journal Title

      Proc. of IJCNN (Int'l Conf. on Neural Networks) (CD-ROM)

      Pages: 2512-2517

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Acquisition of Box Pushing by Direct-Vision-Based Reinforcement Learning2003

    • Author(s)
      Katsunari Shibata, Masaru Iida
    • Journal Title

      Proc. of SICE Annual Conf. 2003 (CD-ROM)

      Pages: 1378-1383

    • NAID

      130005440403

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Direct-Vision-Based強化学習によるカラー物体到達の学習と内部表現2003

    • Author(s)
      幸和芳, 杉坂政典, 柴田克成
    • Journal Title

      第22回計測自動制御学会九州支部学術講演会予稿集

      Pages: 65-68

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 強化学習に基づくコミュニケーションの発現と伝達情報の解析2003

    • Author(s)
      仲西賢展, 杉坂政典, 柴田克成
    • Journal Title

      Proc. of SICE Annual Conf. 2003

      Pages: 71-74

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] メモリQ学習 -強化学習による選択的記憶の獲得-2003

    • Author(s)
      柴田克成, 宮本沢巳
    • Journal Title

      第22回計測自動制御学会九州支部学術講演会予稿集

      Pages: 57-60

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Growing Neural Network for Acquisition of 2-layer Structure2003

    • Author(s)
      Ryusuke Kurino, Masanori Sugisaka, Katsunari Shibata
    • Journal Title

      Proc. of UCNN (Int'l Conf. on Neural Networks) 2003 1455-703(pdf)

      Pages: 2512-2517

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Acquisition of Box Pushing by Direct-Vision-Based Reinforcement Learning2003

    • Author(s)
      Katsunari Shibata Masaru Iida
    • Journal Title

      Proc. of SICE Annual Conf. 2003 0324(pdf)

      Pages: 1378-1383

    • NAID

      130005440403

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Adaptive Space Reconstruction on Hidden Layer and Knowledge Transfer based on Hidden-level Generalization in Layered Neural network

    • Author(s)
      Katsunari Shibata, Koji Ito
    • Journal Title

      Trans. of SICE Vol.43, No.1

      Pages: 54-63

    • NAID

      10018479914

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Book] ニューラルネットワーク計算知能(第12章ニューラルネットを用いた強化学習とロボットの知能)2006

    • Author(s)
      柴田克成, 岡部洋一, 伊藤宏司
    • Total Pages
      22
    • Publisher
      森北出版
    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Annual Research Report 2006 Final Research Report Summary
  • [Publications] K.Shibata, M.Nakanishi: "Discretization of Analog Communication Signals by Noise Addition in Communication Learning"Proc.of The 9th AROB (Int'l Sympo.on Artificial Life and Robotics). Vol.2. 351-354 (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] R.Kurino, M.Sugisaka, K.Shibata: "Growing Neural Network with Hidden Neurons"Proc.of The 9th AROB (Int'l Sympo.on Artificial Life and Robotics). Vol.1. 144-147 (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] K.Yuki, M.Sugisaka, K.Shibata: "Learning of Reaching a Colored Object Based on Direct-Vision-Based Reinforcement Learning and Acquired Internal Representation"Proc.of The 9th AROB (Int'l Sympo.on Artificial Life and Robotics). Vol.2. 486-489 (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] M.Nakanishi, M.Sugisaka, K.Shibata: "Occurrence of State Confusion in the Learning of Communication Using Q-leaning"Proc.of The 9th AROB (Int'l Sympo.on Artificial Life and Robotics). Vol.2. 663-666 (2004)

    • Related Report
      2003 Annual Research Report
  • [Publications] R.Kurino, M.Sugisaka, K.Shibata: "Growing Neural Network for Acquisition of 2-layer Structure"Proc.of IJCNN (Int'l Conf.on Neural Networks)2003. (CD-ROM). 2512-2517 (2003)

    • Related Report
      2003 Annual Research Report
  • [Publications] 柴田克成: "コミュニケーションの強化学習におけるノイズ付加による連続値信号の離散化"電子情報通信学会技術研究報告. Vol.103, No.734. 55-60 (2004)

    • Related Report
      2003 Annual Research Report

URL: 

Published: 2003-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi