Advancement of reinforcement learning and its applications to mobile robots based on spatio-temporal segmentation of the environment

• Project/Area Number: 18500175
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Sensitivity informatics/Soft computing
• Research Institution: Kyushu Institute of Technology
• Principal Investigator: ISHIKAWA Masumi Kyushu Institute of Technology, 大学院・生命体工学研究科, 教授 (60222973)
• Co-Investigator(Kenkyū-buntansha): ZHANG Hong 九州工業大学, 大学院・生命体工学研究科, 助教 (30235709)
• Project Period (FY): 2006 – 2008
• Project Status: Completed (Fiscal Year 2008)
• Budget Amount: ¥4,290,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥690,000); Fiscal Year 2008: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2007: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000); Fiscal Year 2006: ¥1,300,000 (Direct Cost: ¥1,300,000)
• Keywords: 自己組織化 / 強化学習 / 知能ロボティクス / 分節化

Research Abstract

追跡・逃避ゲームを対象とし、モジュール型ネットワーク自己組織化マップを用いたダイナミクス学習層により追跡逃避者間の距離の時間変化を学習し、追跡者・逃避者の最適制御をモジュール化した強化学習層により学習するという、統合的アーキテクチャを提案しその有効性を実証した。また環境が変化した場合、地図の各グリッドが占有されている、あるいは空であるという仮説の下での、センサー系列のそれぞれの尤度を求めることにより、環境変化を検出する方法を提案し、その有効性を実証した。

Research Products

• [Journal Article] Curiosity and Boredom Based on Prediction Error as Novel Internal Rewards (2009)
  • Author(s): Naoyuki Yamamoto, Masumi Ishikawa
  • Journal Title: Brain-Inspired Information Technology Vol. 266 Pages: 51-55
  • Peer Reviewed
• [Journal Article] Classification and Novelty Detection of Omni-view Images Taken from a Mobile Robot (2009)
  • Author(s): Fumiko Kiriake, Masumi Ishikawa
  • Journal Title: Brain-Inspired Information Technology Vol. 266 Pages: 45-49
  • Peer Reviewed
• [Journal Article] Emergence of Behaviors by Reinforcement Learning Based on the Desire for Existence (2009)
  • Author(s): Takao Hagiwara, Masumi Ishikawa
  • Journal Title: Brain-Inspired Information Technology Vol. 266 Pages: 39-44
  • Peer Reviewed
• [Journal Article] Brain-Inspired Emergence of Behaviors Based on Values and Curiosity in Mobile Robots (2009)
  • Author(s): Masumi Ishikawa, 他9名
  • Journal Title: Brain-Inspired Information Technology Vol. 266 Pages: 33-38
  • Peer Reviewed
• [Journal Article] Task segmentation in a mobile robot by mnSOM and clustering with spatio-temporal contiguity (2009)
  • Author(s): Muhammad Aziz Muslim, Masumi Ishikawa, Tetsuo Furukawa
  • Journal Title: International Journal of Innovative Computing, Information & Control Vol. 5, No. 4 Pages: 865-875
  • NAID: 120002441406
  • Peer Reviewed
• [Journal Article] A Fast Stereo Matching Algorithm for Sewer Inspection Robots (2008)
  • Author(s): Alireza Ahrary, Masumi Ishikawa
  • Journal Title: IEEJ Transactions on Electrical and Electronic Engineering Vol. 3, Issue 4 Pages: 441-448
  • NAID: 10025773238
  • Peer Reviewed
• [Journal Article] A Fast Stereo Matching Algorithm for Sewer Inspection Robots (2008)
  • Author(s): Alireza Ahrary, Masumi Ishikawa
  • Journal Title: IEEJ Transactions on Electrical and Electronic Engineering 3 Pages: 441-448
  • NAID: 10025773238
  • Peer Reviewed
• [Journal Article] Prediction of the optimal parameter values in reinforcement learning as a function of the environment (2007)
  • Author(s): K. Kamei, M. Ishikawa
  • Journal Title: Brain-Inspired IT III Vol. 1301 Pages: 210-213
  • Peer Reviewed
• [Journal Article] Training expert modules for a mobile robot using mnSOM (2007)
  • Author(s): M. A. Muslim, M. Ishikawa, T. Furukawa
  • Journal Title: Brain-Inspired IT III Vol. 1301 Pages: 172-175
  • NAID: 120002441380
  • Peer Reviewed
• [Journal Article] Brain-inspired emergence of behaviors in mobile robots (2007)
  • Author(s): M. Ishikawa, K. Ishii, T. Yamakawa, M. Nagamatsu, H. Nakagawa, H. Zhang, T. Furukawa
  • Journal Title: Brain-Inspired IT III Vol. 1301 Pages: 48-51
  • Peer Reviewed
• [Journal Article] Modular network self-organizing map : Can it be an artificial cortex? (2007)
  • Author(s): T. Furukawa, K. Tokunaga, S. Yasui, H. Tamukoh, K. Ishii, M. Ishikawa, K. Horio, K. Natsume
  • Journal Title: Brain-Inspired IT III Vol. 1301 Pages: 43-47
  • Peer Reviewed
• [Journal Article] Task Segmentation in a Mobile Robot by mnSOM : A New Approach To Training Expert Modules (2007)
  • Author(s): M. Aziz Muslim, Masumi Ishikawa, Tetsuo Furukawa
  • Journal Title: Neural Computing and Applications Vol. 16, Issue 6 Pages: 571-580
  • NAID: 120002441381
  • Peer Reviewed
• [Journal Article] Navigation of an autonomous sewer inspection robots based on stereo camera images and laser scanner data (2007)
  • Author(s): Alireza Ahrary, Li Tian, Sei-Ichiro Kamata, Masumi Ishikawa
  • Journal Title: International Journal on Artificial Intelligence Tools (IJAIT) Vol. 16, Issue 4 Pages: 611-625
  • NAID: 120002441404
  • Peer Reviewed
• [Journal Article] An automated intelligent fault detection system for inspection of sewer pipes (2007)
  • Author(s): Alireza Ahrary, Yoshinori Kawamura, Masumi Ishikawa
  • Journal Title: 電気学会論文誌C Vol. 127, No. 6 Pages: 943-950
  • NAID: 130000090473
  • Peer Reviewed
• [Journal Article] A study on an autonomous mobile robot for sewer inspection system (2007)
  • Author(s): A. Ahrary, A. A. F Nassiraei, M. Ishikawa
  • Journal Title: Journal of Artificial Life and Robotics Vol. 11, no. 1 Pages: 23-27
  • Peer Reviewed
• [Journal Article] A study of an autonomous mobile robot for sewer inspection system (2007)
  • Author(s): A. Ahrary, A. Nassiraei, M. Ishikawa
  • Journal Title: Artificial Life and Robotics Vol. 11, No. 1 Pages: 23-27
  • Peer Reviewed
• [Journal Article] Task Segmentation in a Mobile Robot by mnSOM: A New Approach To Training Expert Modules (2007)
  • Author(s): M.Aziz Muslim
  • Journal Title: Neural Computing and Applications 16-6 Pages: 571-580
  • Peer Reviewed
• [Journal Article] Brain-inspired emergence of behaviors in mobile robots (2007)
  • Author(s): M.Ishikawa
  • Journal Title: Brain-Inspired IT 1301 Pages: 48-51
  • Peer Reviewed
• [Journal Article] Prediction of the optimal parameter values in reinforcement learning as a funcation of the environment (2007)
  • Author(s): K.Kamei
  • Journal Title: Brain-Inspired IT 1301 Pages: 210-213
  • Peer Reviewed
• [Journal Article] Trends (2007)
  • Author(s): Frederik Linaker
  • Journal Title: Neural Computation(Robot Localization Using Vision), Springer Pages: 483-512
• [Journal Article] パラメータの相互依存性を考慮した強化学習の最適パラメータ推定 (2007)
  • Author(s): 亀井圭史, 石川眞澄
  • Journal Title: 電子情報通信学会技術研究報告 Vol.106, No.588, NC2006-150 Pages: 191-196
  • NAID: 110006249049
• [Journal Article] Dependency of values of parameters in reinforcement learning for navigation of a mobile robot on the environment (2006)
  • Author(s): Keiji Kamei, Masumi Ishikawa
  • Journal Title: Neural Information Processing---Letters and Reviews Vol. 10, Nos. 7-9 Pages: 219-226
  • Peer Reviewed
• [Journal Article] A New Approach to Task Segmentation in Mobile Robots by mnSOM (2006)
  • Author(s): Muhammad Aziz Muslim, Masumi Ishikawa, Tetsuo Furukawa
  • Journal Title: IEEE World Congress on Computational Intelligence, Vancouver, Canada Pages: 6542-6549
• [Journal Article] Training expert modules for a mobile robot using mnSOM (2006)
  • Author(s): Muhammad Aziz Muslim, Masumi Ishikawa, Tetsuo Furukawa
  • Journal Title: Abstracts of BrainIT2006 (The Third International Conference on Brain-Inspired Information Technology) Pages: 73-73
  • NAID: 120002441380
• [Journal Article] Prediction of the optimal values of parameters in reinforcement learning as a function of the environment (2006)
  • Author(s): Keiji Kamei, Masumi Ishikawa
  • Journal Title: Abstracts of BrainIT2006 (The Third International Conference on Brain-Inspired Information Technology) Pages: 85-85
• [Journal Article] Dependency of values of parameters in reinforcement learning for navigation of a mobile robot on the environment (2006)
  • Author(s): Keiji Kamei, Masumi Ishikawa
  • Journal Title: Neural Information Processing---Letters and Reviews Vol.10, No.7-9 Pages: 219-226
• [Presentation] 知性獲得への自己組織的アプローチ (2010)
  • Author(s): 石川眞澄
  • Organizer: 第11回自己組織化マップ研究会2010講演論文集
  • Place of Presentation: 北九州
  • Year and Date: 2010-03-15
• [Presentation] 強化学習を用いた変動環境下の最短経路探索 (2010)
  • Author(s): 甲斐孝史, 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2010-03-09
• [Presentation] 進化型モジュールネットワークとモジュール型強化学習を用いた階層型アーキテクチャ (2010)
  • Author(s): 金源直幸, 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2010-03-09
• [Presentation] 画像特徴抽出フィルタの学習に基づくロボットの位置推定 (2010)
  • Author(s): 大木万里子, 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2010-03-09
• [Presentation] Dual Map Building and Localization for Mobile Robots Based on Panoramic Images (2010)
  • Author(s): Youbo Cai, Masumi Ishikawa
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2010-03-09
• [Presentation] Origins of Intelligence--Segmentation and Curiosity (2009)
  • Author(s): Masumi Ishikawa
  • Organizer: ICONIP2009
  • Place of Presentation: Bangkok
  • Year and Date: 2009-12-02
• [Presentation] Hierarchical architecture with modular network SOM and modular reinforcement learning (2009)
  • Author(s): Masumi Ishikawa, Kosuke Ueno
  • Organizer: ICANN 2009, LNCS 5768
  • Place of Presentation: Limassol, Cyprus
  • Year and Date: 2009-09-15
• [Presentation] Reinforcement learning with internal rewards based on error in a grid-based map (2009)
  • Author(s): Masumi Ishikawa, Yoshifumi Tanaka
  • Organizer: IMS2009
  • Place of Presentation: Kunming, China
  • Year and Date: 2009-07-22
• [Presentation] グリッドベース地図誤差に基づく好奇心を内部報酬とする強化学習 (2009)
  • Author(s): 田中芳典, 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2009-03-13
• [Presentation] 強化学習を用いた生存欲に基づく行動の創発 (2009)
  • Author(s): 守田観輝夫, 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2009-03-13
• [Presentation] 強化学習を用いた生存欲に基づく行動の創発 (2009)
  • Author(s): 守田観輝夫, 石川波澄
  • Organizer: 電子情報通信学会ニューロコンピューティング研究会
  • Place of Presentation: 東京都町田市
  • Year and Date: 2009-03-13
• [Presentation] グリッドペース地図誤差に基づく好奇心を内部報酬とする強化学習 (2009)
  • Author(s): 田中芳典, 石川眞澄
  • Organizer: 電子情報通信学会ニューロコンピューティング研究会
  • Place of Presentation: 東京都町田市
  • Year and Date: 2009-03-13
• [Presentation] 拡散的好奇心付き標準的粒子群最適化器の効果 (2009)
  • Author(s): 章宏, 石川眞澄
  • Organizer: 電子情報通信学会ニューロコンピューティング研究会
  • Place of Presentation: 東京都町田市
  • Year and Date: 2009-03-12
• [Presentation] 知能の源-----分節化および好奇心 (2009)
  • Author(s): 石川眞澄, 蔡幼波
  • Organizer: Brain-ISワークショップ
  • Place of Presentation: 北九州
  • Year and Date: 2009-02-04
• [Presentation] 知能の根源としての分節化と好奇心 (2008)
  • Author(s): 石川眞澄
  • Organizer: 日本知能情報ファジィ学会合同ワークショップ
  • Place of Presentation: 福岡
  • Year and Date: 2008-12-06
• [Presentation] 知能の根源としての分節化と好奇心 (2008)
  • Author(s): 石川眞澄
  • Organizer: 日本知能情報ファジィ学会合同ワークショップ
  • Place of Presentation: 福岡市
  • Year and Date: 2008-12-06
• [Presentation] Segmentation and curiosity as origins of intelligence (2008)
  • Author(s): Masumi Ishikawa
  • Organizer: International Workshop on Hybrid and Adaptive Systems for Real-time Robotics Vision and Control, Auckland, New Zealand
  • Place of Presentation: Keynote Speech
  • Year and Date: 2008-11-28
• [Presentation] Segmentation and curiosity as origins of intelligence(Keynote Speech) (2008)
  • Author(s): Masumi Is
  • Organizer: Int. Workshop on Hybrid and Adaptive Systems for Real-time Robotics Vision
  • Place of Presentation: オークランド
  • Year and Date: 2008-11-28
• [Presentation] Brain-inspired emergence of behaviors based on the desire for existence by (2008)
  • Author(s): Masumi Ishikawa, Mikio Morita
  • Organizer: reinforcement learning
  • Place of Presentation: New Zealand
  • Year and Date: 2008-11-26
• [Presentation] Brain-inspired emergence of behaviors based on the desire for existence by reinforcement learning (2008)
  • Author(s): Masumi Ishikawa, Mikio Morita
  • Organizer: International Conference on Neural Information Processing (ICONIP2008)
  • Place of Presentation: オークランド
  • Year and Date: 2008-11-26
• [Presentation] Brain-inspired emergence of behaviors in mobile robots by reinforcement learning with internal rewards (2008)
  • Author(s): Masumi Ishikawa, Takao Hagiwara, Naoyuki Yamamoto, Fumiko Kiriake
  • Organizer: 8th Internationa Conference on Hybrid Intelligent Systems (HIS2008)
  • Place of Presentation: Barcelona, Spain
  • Year and Date: 2008-09-10
• [Presentation] Brain-inspired emergence of behaviors in mobile robots by reinforcement learning with internal rewards (2008)
  • Author(s): M. Ishikawa, T. Hagiwara, N. Yamamoto, F. Kiriake
  • Organizer: 8th Internationa Conference on Hybrid Intelligent Systems(HIS2008)
  • Place of Presentation: バルセロナ
  • Year and Date: 2008-09-10
• [Presentation] Evolutionary Canonical Particle Swarm Optimizer --? A Proposal of Meta-Optimization in Model Selection (2008)
  • Author(s): Hong Zhang, Masumi Ishikawa
  • Organizer: 18th International Conference on Arfiticial Neural Networks (ICANN2008)
  • Place of Presentation: プラハ
  • Year and Date: 2008-09-05
• [Presentation] Brain-inspired Emergence of Behaviors by Reinforcement Learning (2008)
  • Author(s): Masumi Ishikawa, Mikio Morita, Takao Hagiwara
  • Organizer: 8th Postech-Kyutech Joint Workshop on Neuroinformatics
  • Place of Presentation: 北九州
  • Year and Date: 2008-08-26
• [Presentation] 忘却から好奇心へ (2008)
  • Author(s): 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 沖縄
  • Year and Date: 2008-06-26
• [Presentation] フェロー記念講演「忘却から好奇心へ」 (2008)
  • Author(s): 石川眞澄
  • Organizer: 電子情報通信学会ニューロコンピューティング研究会
  • Place of Presentation: 沖縄
  • Year and Date: 2008-06-26
• [Presentation] Formation of Graph-based Maps for Mobile Robots using Hidden Markov Models (2008)
  • Author(s): M. Aziz Muslim, Masumi Ishikawa
  • Organizer: 2008 IEEE World Congress on Computational Intelligence (WCCI2008)
  • Place of Presentation: Hong Kong
  • Year and Date: 2008-06-03
• [Presentation] Formation of Graph-based Maps for Mobile Robots using Hidden Markov Models (2008)
  • Author(s): M. Aziz Muslim, Masumi Ishikawa
  • Organizer: 2008 IEEE World Congress on Computational Intelligence (WCCI2008)
  • Place of Presentation: 香港
  • Year and Date: 2008-06-03
• [Presentation] 価値および好奇心に基づく移動ロボットの脳型行動創発 (2008)
  • Author(s): 石川眞澄, 永松正博, 萩原隆男, 山本尚幸, 桐明芙三子, 西田武彦, 山川烈, 石井和男, 中川秀樹, 章宏
  • Organizer: 電子情報通信学会総合大会
  • Place of Presentation: 北九州
  • Year and Date: 2008-03-19
• [Presentation] Graph-based Maps Formation for Mobile Robots by Hidden Markov Models (2008)
  • Author(s): Muhammad Aziz Muslim, Masumi Ishikawa
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2008-03-13
• [Presentation] 内発的動機付けを用いた移動ロボットの行動計画 (2008)
  • Author(s): 山本尚幸, 石川眞澄
  • Organizer: 電子情報通信学会技術研究報告
  • Place of Presentation: 東京
  • Year and Date: 2008-03-13
• [Presentation] Task Segmentation in a Mobile Robot by mnSOM and Clustering with Spatio-temporal Contiguity (2007)
  • Author(s): Muhammad Aziz Muslim, Masumi Ishikawa, Tetsuko Furukawa
  • Organizer: ICONIP2007
  • Year and Date: 2007-11-16
• [Presentation] Task Segmentation in a Mobile Robot by mnSOM and Clustering with Spatio-temporal Contiguity (2007)
  • Author(s): M.Aziz Muslim
  • Organizer: ICONIP2007
  • Place of Presentation: 北九州市
  • Year and Date: 2007-11-16
• [Presentation] Task Segmentation in a Mobile Robot by mnSOM and Hierarchical Clustering (2007)
  • Author(s): M. Aziz Muslim, Masumi Ishikawa, Tetsuo Furukawa
  • Organizer: Proceedings of the 6th International Workshop on Self-Ogranizing Maps (WSOM2007)
  • Place of Presentation: Bielefeld, Germany
  • Year and Date: 2007-09-05
• [Presentation] Emergence of Behaviors Based on the Desire for Existence (2007)
  • Author(s): Masumi Ishikawa, Takao Hagiwara
  • Organizer: Proceedings of the 7th Postech-Kyutech Joint Workshop on Neuroinformatics
  • Place of Presentation: Pohang, Korea
  • Year and Date: 2007-08-27
• [Presentation] A New Approach to Task Segmentation in Mobile Robots by mnSOM (2006)
  • Author(s): Muhammad Aziz Muslim, Masumi Ishikawa, Tetsuo Furukawa
  • Organizer: IEEE World Congress on Computational Intelligence
  • Place of Presentation: Vancouver, Canada
  • Year and Date: 2006-07-19
• [Book] Trends in Intelligent Systems and Computer Engineering (2008)
  • Author(s): Hong Zhang, Masumi Ishikawa
  • Total Pages: 16
  • Publisher: Springer
• [Book] Trends in Neural Computation(Robot Localization Using Vision(483-512)) (2007)
  • Author(s): Frederik Linaker, Masumi Ishikawa
• [Book] Trends in Neural Computation(Robot Localization Using Vision) (2007)
  • Author(s): Frederik Linaker
  • Publisher: Springer
• [Book] Brain-Inspired IT II (Reduction of computational cost in optimization of parameter values in reinforcement learning by a genetic algorithm) (2006)
  • Author(s): Keiji Kamei, Masumi Ishikawa
  • Total Pages: 292
  • Publisher: Elsevier
• [Remarks] ホームページ等
  • URL: http://www.brain.kyutech.ac.jp/~ishikawa
• [Remarks]
  • URL: http://www.brain.kyutech.ac.jp/~ishikawa
• [Remarks]
  • URL: http://www.brain.kyutech.ac.jp/~ishikawa