• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Exploration of a Breakthrough Technology for Emergence of Symbol Processing by Neuro-based Reinforcement Learning Considering Time Axis

Research Project

Project/Area Number 23500245
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Perception information processing/Intelligent robotics
Research InstitutionOita University

Principal Investigator

SHIBATA Katsunari  大分大学, 工学部, 准教授 (10260522)

Project Period (FY) 2011 – 2013
Project Status Completed (Fiscal Year 2013)
Budget Amount *help
¥5,200,000 (Direct Cost: ¥4,000,000、Indirect Cost: ¥1,200,000)
Fiscal Year 2013: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2012: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2011: ¥2,990,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥690,000)
Keywords知能ロボット / 強化学習 / ニューラルネット / 高次機能 / シンボル処理創発 / 因果トレース / 概念形成 / コミュニケーション学習 / リカレントニューラルネット / ロボット / 動詞表現 / 離散的・抽象的表現 / 自律的役割分担 / 知能創発 / 微分型トレース / 時間軸調整 / 動詞表現獲得 / 時間軸 / コミュニケーション / 自律学習
Research Abstract

Efficient learning in a huge amount of spatio-temporal information holds the key to the emergence of higher functions in the real world. In this research, handling of time axis was especially focused on. A novel idea named "Causality traces" is propounded which judge the importance of events "subjectively" and are used for retrospective learning. Its learning performance exceeds that with the conventional method in value learning. Next, based on the idea that "concept" is formed from the difference of necessary motions, it is confirmed that discrete and abstract internal state representations are autonomously formed in a recurrent neural network through reinforcement learning. Furthermore, in autonomous communication learning, it was shown that information about target movement could be transmitted after learning. A novel perspective could be introduced to the handling of the time axis, but for the emergence of symbol processing, learning of dynamics should be further improved.

Report

(4 results)
  • 2013 Annual Research Report   Final Research Report ( PDF )
  • 2012 Research-status Report
  • 2011 Research-status Report
  • Research Products

    (45 results)

All 2014 2013 2012 2011

All Journal Article (25 results) (of which Peer Reviewed: 7 results) Presentation (20 results) (of which Invited: 1 results)

  • [Journal Article] 因果トレース - 並列かつ主観的時間スケールの導入による過去の処理の効率的学習 -2014

    • Author(s)
      柴田 克成
    • Journal Title

      電子情報通信学会技術報告

      Volume: NC2013 Pages: 157-162

    • Related Report
      2013 Annual Research Report
  • [Journal Article] Emergence of Flexible Prediction-Based Discrete Decision Making and Continuous Motion Generation through Actor-Q-Learning2013

    • Author(s)
      Katsunari Shibata and Kenta Goto
    • Journal Title

      Proc. of Int'l Conf. on Development and Learning and on Epigenetic Robotics (ICDL-Epirob)

      Volume: ID 15 (CDROM)

    • Related Report
      2013 Final Research Report
  • [Journal Article] Emergence of Flexible Prediction-Based Discrete Decision Making and Continuous Motion Generation through Actor-Q-Learning2013

    • Author(s)
      Katsunari Shibata and Kenta Goto
    • Journal Title

      Proc. of Int'l Conf. on Development and Learning and on Epigenetic Robotics (ICDL-Epirob) 2013

    • Related Report
      2013 Annual Research Report
    • Peer Reviewed
  • [Journal Article] 強化学習によるリカレントニューラルネットワーク内部での振動子創発の可能性2013

    • Author(s)
      品矢 裕介, 柴田克成
    • Journal Title

      第32回計測自動制御学会九州支部学術講演会予稿集

      Pages: 74-77

    • Related Report
      2013 Annual Research Report
  • [Journal Article] RNNを用いた強化学習によるセンサ信号の時間変化を表すコミュニケーションの創発2013

    • Author(s)
      朱 祺, 柴田克成
    • Journal Title

      第32回計測自動制御学会九州支部学術講演会予稿集

      Pages: 71-74

    • Related Report
      2013 Annual Research Report
  • [Journal Article] 予測を要して連続動作を含む柔軟な行動のActor-Q学習による獲得2013

    • Author(s)
      柴田克成, 後藤健太
    • Journal Title

      第23回インテリジェント・システム・シンポジウム (FAN 2013) 講演論文集

      Pages: 86-91

    • Related Report
      2013 Annual Research Report
  • [Journal Article] ニューラルネットを用いた強化学習による行動の学習を通した色恒常性の創発2013

    • Author(s)
      柴田克成, 栗崎俊介
    • Journal Title

      電子情報通信学会技術報告

      Volume: NC2012-134 - NC2012-182 Pages: 215-220

    • Related Report
      2012 Research-status Report
  • [Journal Article] Mohamad Faizal bin Samsudin and Katsunari Shibata, Emergence of Discrete and Abstract State Representation in Continuous Input Task through Reinforcement Learning, Advances in Intelligent Systems and Computing, Robot Intelligence Technology and Applications 20122012

    • Author(s)
      Yoshito Sawatsubashi, Mohamad Faizal bin Samsudin and Katsunari Shibata
    • Journal Title

      Proc. of RiTA

      Volume: 13-22

    • Related Report
      2013 Final Research Report
  • [Journal Article] Differential Trace in Learning of Value Function with a Neural Network, Advances in Intelligent Systems and Computing, Robot Intelligence Technology and Applications 20122012

    • Author(s)
      Katsunari Shibata and Shuji Enoki
    • Journal Title

      Proc. of RiTA

      Volume: 55-64

    • Related Report
      2013 Final Research Report
  • [Journal Article] Emergence of Multi-Step Discrete State Transition through Reinforcement Learning with a Recurrent Neural Network2012

    • Author(s)
      Mohamad Faizal bin Samsudin, Yoshito Sawatsubashi and Katsunari Shibata
    • Journal Title

      LNCS(Lecture Notes in Computer Science), Neural Information Processing, Proc. of ICONIP (Int'l Conf. on Neural Information Processing)

      Volume: 7664 Pages: 583-590

    • Related Report
      2013 Final Research Report
  • [Journal Article] Emergence of Color Constancy Illusion through Reinforcement Learning with a Neural Network2012

    • Author(s)
      Katsunari Shibata and Shunsuke Kurizaki
    • Journal Title

      Proc. of ICDL-EpiRob (Int'l Conf. on Development and Learning- Epigenetic Robotics)

      Pages: 1-6

    • DOI

      10.1109/devlrn.2012.6400580

    • Related Report
      2013 Final Research Report 2012 Research-status Report
  • [Journal Article] Differential Trace in Learning of Value Function with a Neural Network2012

    • Author(s)
      Katsunari Shibata and Shuji Enoki
    • Journal Title

      Advances in Intelligent Systems and Computing, Robot Intelligence Technology and Applications 2012, Proc. of RiTA 2012

      Volume: 1 Pages: 55-64

    • Related Report
      2012 Research-status Report
    • Peer Reviewed
  • [Journal Article] Emergence of Discrete and Abstract State Representation in Continuous Input Task through Reinforcement Learning2012

    • Author(s)
      Yoshito Sawatsubashi, Mohamad Faizal bin Samsudin and Katsunari Shibata
    • Journal Title

      Advances in Intelligent Systems and Computing, Robot Intelligence Technology and Applications 2012, Proc. of RiTA 2012

      Volume: 1 Pages: 13-22

    • Related Report
      2012 Research-status Report
    • Peer Reviewed
  • [Journal Article] Emergence of Multi-Step Discrete State Transition through Reinforcement Learning with a Recurrent Neural Network2012

    • Author(s)
      Mohamad Faizal bin Samsudin, Yoshito Sawatsubashi and Katsunari Shibata
    • Journal Title

      LNCS(Lecture Notes in Computer Science), Neural Information Processing, Proc. of ICONIP2012

      Volume: 1 Pages: 583-590

    • Related Report
      2012 Research-status Report
    • Peer Reviewed
  • [Journal Article] 強化学習による 合目的的かつ接地した一方向コミュニケーションの創発2012

    • Author(s)
      柴田克成, 笹原冬月
    • Journal Title

      計測自動制御学会 システム・情報部門 学術講演会 講演論文集

      Volume: 1 Pages: 390-395

    • Related Report
      2012 Research-status Report
  • [Journal Article] ニューラルネットを用いた価値関数の学習における微分型トレースの提案2012

    • Author(s)
      榎修志, 柴田克成
    • Journal Title

      計測自動制御学会 システム・情報部門 学術講演会 講演論文集

      Volume: 1 Pages: 396-401

    • Related Report
      2012 Research-status Report
  • [Journal Article] リカレントネットを用いた強化学習における離散的かつ抽象的な状態表現の創発2012

    • Author(s)
      沢津橋由人, Mohamad Faizal Bin Samusudin, 柴田克成
    • Journal Title

      計測自動制御学会 システム・情報部門 学術講演会 講演論文集

      Volume: 1 Pages: 402-407

    • Related Report
      2012 Research-status Report
  • [Journal Article] Emergence of Purposive and Grounded Communication through Reinforcement Learning, LNCS(Lecture Notes in Computer Science)2011

    • Author(s)
      Katsunari Shibata and Kazuki Sasahara
    • Journal Title

      LNCS(Lecture Notes in Computer Science), Vol. 7064, Proc. of ICONIP (Int'l Conf. on Neural Information Processing)

      Volume: 7064 Pages: 66-75

    • Related Report
      2013 Final Research Report
  • [Journal Article] Discovery of Pattern Meaning from Delayed ...2011

    • Author(s)
      Katsunari Shibata & Hiroki Utsunomiya
    • Journal Title

      Proc. of Int'l Joint Conf. on Neural Networks 2011

      Volume: - Pages: 1445-1452

    • Related Report
      2011 Research-status Report
    • Peer Reviewed
  • [Journal Article] Discount and Speed/Execution tradeoffs in ...2011

    • Author(s)
      R. Uribe, F. Lozanom, K. Shibata & C. Anderson
    • Journal Title

      Proc. of IEEE Conf. on CIG 2011

      Volume: - Pages: 79-86

    • Related Report
      2011 Research-status Report
    • Peer Reviewed
  • [Journal Article] Emergence of Purposive and Grounded ...2011

    • Author(s)
      Katsunari Shibata and Kazuki Sasahara
    • Journal Title

      LNCS(Lecture Notes in Computer Science)

      Volume: Vol. 7064 Pages: 66-75

    • Related Report
      2011 Research-status Report
    • Peer Reviewed
  • [Journal Article] リカレントネットを用いた強化学習による探索行動と多値記憶の創発2011

    • Author(s)
      柴田克成, 後藤健太
    • Journal Title

      電子情報通信学会技術報告

      Volume: -

    • NAID

      110009546290

    • Related Report
      2011 Research-status Report
  • [Journal Article] Context-based Word Recognition through ...2011

    • Author(s)
      Ahmad Afif Mohd Faudzi & Katsunari Shibata
    • Journal Title

      第30回SICE九州支部学術講演会予稿集

      Volume: - Pages: 155-158

    • Related Report
      2011 Research-status Report
  • [Journal Article] 画像を入力とするニューラルネットの学習における方位選択性入力の付加2011

    • Author(s)
      沢津橋由人, 柴田克成
    • Journal Title

      第30回SICE九州支部学術講演会予稿集

      Volume: - Pages: 151-154

    • Related Report
      2011 Research-status Report
  • [Journal Article] リカレントネットによる内部状態遷移を要する問題学習時の初期重み値の影響2011

    • Author(s)
      田口優馬, 柴田克成
    • Journal Title

      第30回SICE九州支部学術講演会予稿集

      Volume: - Pages: 87-90

    • Related Report
      2011 Research-status Report
  • [Presentation] 因果トレース - 並列かつ主観的時間スケールの導入による過去の処理の効率的学習2014

    • Author(s)
      柴田克成
    • Organizer
      電子情報通信学会ニューロコンピューティング研究会
    • Place of Presentation
      東京
    • Year and Date
      2014-03-18
    • Related Report
      2013 Final Research Report
  • [Presentation] 因果トレース - 並列かつ主観的時間スケールの導入による過去の処理の効率的学習 -2014

    • Author(s)
      柴田克成
    • Organizer
      電子情報通信学会ニューロコンピューティング研究会
    • Place of Presentation
      玉川大学,東京都
    • Related Report
      2013 Annual Research Report
  • [Presentation] 強化学習によるリカレントニューラルネットワーク内部での振動子創発の可能性2013

    • Author(s)
      品矢 裕介
    • Organizer
      第32回計測自動制御学会九州支部学術講演会
    • Place of Presentation
      長崎
    • Year and Date
      2013-12-01
    • Related Report
      2013 Final Research Report
  • [Presentation] RNN を用いた強化学習によるセンサ信号の時間変化を表すコミュニケーションの創発2013

    • Author(s)
      朱 祺
    • Organizer
      第32回計測自動制御学会九州支部学術講演会
    • Place of Presentation
      長崎
    • Year and Date
      2013-12-01
    • Related Report
      2013 Final Research Report
  • [Presentation] 強化学習によるリカレントニューラルネットワーク内部での振動子創発の可能性2013

    • Author(s)
      品矢 裕介, 柴田克成
    • Organizer
      第32回計測自動制御学会九州支部学術講演会
    • Place of Presentation
      長崎大学,長崎県
    • Related Report
      2013 Annual Research Report
  • [Presentation] RNNを用いた強化学習によるセンサ信号の時間変化を表すコミュニケーションの創発2013

    • Author(s)
      朱 祺, 柴田克成
    • Organizer
      第32回計測自動制御学会九州支部学術講演会
    • Place of Presentation
      長崎大学,長崎県
    • Related Report
      2013 Annual Research Report
  • [Presentation] 予測を要して連続動作を含む柔軟な行動のActor-Q学習による獲得2013

    • Author(s)
      柴田克成, 後藤健太
    • Organizer
      第23回インテリジェント・システム・シンポジウム (FAN 2013)
    • Place of Presentation
      九州大学,福岡県
    • Related Report
      2013 Annual Research Report
  • [Presentation] Emergence of Flexible Prediction-Based Discrete Decision Making and Continuous Motion Generation through Actor-Q-Learning2013

    • Author(s)
      Katsunari Shibata and Kenta Goto
    • Organizer
      Int'l Conf. on Development and Learning and on Epigenetic Robotics (ICDL-Epirob) 2013
    • Place of Presentation
      Osaka, Japan
    • Related Report
      2013 Annual Research Report
  • [Presentation] ニューラルネットを用いた強化学習による行動の学習を通した色恒常性の創発2013

    • Author(s)
      ○柴田克成, 栗崎俊介
    • Organizer
      電子情報通信学会 ニューロコンピューティング研究会
    • Place of Presentation
      東京都町田市
    • Related Report
      2012 Research-status Report
  • [Presentation] ニューラルネットを用いた価値関数の学習における 微分型トレースの提案2012

    • Author(s)
      榎修志
    • Organizer
      計測自動制御学会システム, 情報部門学術講演会
    • Place of Presentation
      名古屋
    • Year and Date
      2012-11-23
    • Related Report
      2013 Final Research Report
  • [Presentation] リカレントネットを用いた強化学習における 離散的かつ抽象的な状態表現の創発2012

    • Author(s)
      沢津橋由人
    • Organizer
      計測自動制御学会 システム, 情報部門学術講演会
    • Place of Presentation
      名古屋
    • Year and Date
      2012-11-23
    • Related Report
      2013 Final Research Report
  • [Presentation] Differential Trace in Learning of Value Function with a Neural Network2012

    • Author(s)
      Katsunari Shibata and ○Shuji Enoki
    • Organizer
      RiTA (Robot Intelligent Technology and Applications) 2012
    • Place of Presentation
      光州(韓国)
    • Related Report
      2012 Research-status Report
  • [Presentation] Emergence of Discrete and Abstract State Representation in Continuous Input Task through Reinforcement Learning2012

    • Author(s)
      ○Yoshito Sawatsubashi, Mohamad Faizal bin Samsudin and Katsunari Shibata
    • Organizer
      RiTA (Robot Intelligent Technology and Applications) 2012
    • Place of Presentation
      光州(韓国)
    • Related Report
      2012 Research-status Report
  • [Presentation] Emergence of Multi-Step Discrete State Transition through Reinforcement Learning with a Recurrent Neural Network2012

    • Author(s)
      ○Mohamad Faizal bin Samsudin, Yoshito Sawatsubashi and Katsunari Shibata
    • Organizer
      ICONIP (Int'l Conf. on Neural Information Processing Systems) 2012
    • Place of Presentation
      Doha (Qatar)
    • Related Report
      2012 Research-status Report
  • [Presentation] Emergence of Color Constancy Illusion through Reinforcement Learning with a Neural Network2012

    • Author(s)
      ○Katsunari Shibata and Shunsuke Kurizaki
    • Organizer
      Proc. of ICDL-EpiRob (Int'l Conf. on Development and Learning - Epigenetic Robotics) 2012
    • Place of Presentation
      San Diego (USA)
    • Related Report
      2012 Research-status Report
  • [Presentation] 強化学習による 合目的的かつ接地した一方向コミュニケーションの創発2012

    • Author(s)
      ○柴田克成, 笹原冬月
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会
    • Place of Presentation
      名古屋市
    • Related Report
      2012 Research-status Report
  • [Presentation] ニューラルネットを用いた価値関数の学習における微分型トレースの提案2012

    • Author(s)
      ○榎修志, 柴田克成
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会
    • Place of Presentation
      名古屋市
    • Related Report
      2012 Research-status Report
  • [Presentation] リカレントネットを用いた強化学習における離散的かつ抽象的な状態表現の創発2012

    • Author(s)
      ○沢津橋由人, Mohamad Faizal Bin Samusudin, 柴田克成
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会
    • Place of Presentation
      名古屋市
    • Related Report
      2012 Research-status Report
  • [Presentation] あめとむちで知能を作る? -知能ロボットって本当に賢いの?2011

    • Author(s)
      柴田克成
    • Organizer
      SOFT九州支部夏季ワークショップ
    • Place of Presentation
      玉名(熊本県)
    • Year and Date
      2011-09-01
    • Related Report
      2013 Final Research Report
    • Invited
  • [Presentation] あめとむちで知能を作る? ー知能ロボットって...2011

    • Author(s)
      柴田 克成
    • Organizer
      SOFT九州支部夏季ワークショップ2011(招待講演)
    • Place of Presentation
      熊本県玉名市
    • Related Report
      2011 Research-status Report

URL: 

Published: 2011-08-05   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi