Computational model of human decision-making in complicated environments and its applications

• Project/Area Number: 18300101
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Bioinformatics/Life informatics
• Research Institution: Kyoto University (2007-2008); Nara Institute of Science and Technology (2006)
• Principal Investigator: ISHII Shin Kyoto University, 大学院・情報学研究科, 教授 (90294280)
• Co-Investigator(Kenkyū-buntansha): SHIBATA Tomohiro 奈良先端科学技術大学院大学, 情報科学研究科, 准教授 (40359873) ; YOSHIDA Wako University College London, Researcher (30379599) ; NAKAMURA Yutaka 大阪大学, 大学院・工学研究科, 助教 (70403334) ; MORI Takeshi 京都大学, 大学院・情報学研究科, 研究員 (00457144)
• Project Period (FY): 2006 – 2008
• Project Status: Completed (Fiscal Year 2008)
• Budget Amount: ¥17,710,000 (Direct Cost: ¥15,100,000、Indirect Cost: ¥2,610,000); Fiscal Year 2008: ¥4,810,000 (Direct Cost: ¥3,700,000、Indirect Cost: ¥1,110,000); Fiscal Year 2007: ¥6,500,000 (Direct Cost: ¥5,000,000、Indirect Cost: ¥1,500,000); Fiscal Year 2006: ¥6,400,000 (Direct Cost: ¥6,400,000)
• Keywords: 部分観測 / 前頭前野 / 意思決定 / サンプリング / 多自由ロボット / 強化学習 / 計算論的認知心理学 / 多自由度ロボット / モジュール構造 / 教科学習

Research Abstract

線形確率システムとして表現できないような複雑あるいは動的な環境における最適意思決定過程を模擬する機械学習モデルを、強化学習に注目して構築し、工学応用、特に多自由度ロボットに対する自律制御に適用した。高度で階層的な推論を必要とするタスクを題材とし、複雑な問題解決に関わる脳内モデルを構築し、非侵襲脳活動計測器を用いた認知科学実験により検証した。

Research Products

• [Journal Article] Adaptive particle allocation for multi-fo cal visual attention based on particle filtering (2009)
  • Author(s): N,Yano., T,Shibata., S,Ishii
  • Journal Title: Artificial Life and Robotics 13 Pages: 522-525
  • Peer Reviewed
• [Journal Article] Adaptive particle allocation for multifocal visual attention based on particle filtering (2009)
  • Author(s): N. Yano
  • Journal Title: Artificial Life and Robotics 13 Pages: 522-525
  • Peer Reviewed
• [Journal Article] Prediction of aperiodic target sequences by saccades (2008)
  • Author(s): M,Shikauchi., S,Ishii., T,Shibata
  • Journal Title: Behavioural Brain Research 189 Pages: 325-331
  • Peer Reviewed
• [Journal Article] Virtual force/tactile sensors for inter- active machines using user's biological signals (2008)
  • Author(s): T,Tamei., S,Ishii., T,Shibata
  • Journal Title: Advanced Robotics 22 Pages: 893-911
  • Peer Reviewed
• [Journal Article] Collaborative prediction by multiple Bayesian networks and its application to printer usage modeling (2008)
  • Author(s): J,Hirayama., M,Nakatomi., T,Takenouchi., S,ishii
  • Journal Title: Behaviormetrika 35 Pages: 99-114
  • NAID: 110006950237
  • Peer Reviewed
• [Journal Article] A probabilistic modeling of MOSAIC learning (2008)
  • Author(s): S,Osaga., J,Hirayama., T,Takenouchi., S,Ishii
  • Journal Title: Artificial Life and Robotics 12 Pages: 167-171
  • Peer Reviewed
• [Journal Article] Adaptive control of a looper -like robot based on the CPG-actor-critic method (2008)
  • Author(s): K,Makino., Y,Nakamura., T,Shibata., S, Ishii
  • Journal Title: Artificial Life and Robotics 12 Pages: 129-132
  • Peer Reviewed
• [Journal Article] Collaborative prediction by multiple Bayesian networks and its application to printer usage modeling (2008)
  • Author(s): J. Hirayama
  • Journal Title: Behaviormetrika 35 Pages: 99-114
  • Peer Reviewed
• [Journal Article] Virtual force/tactile sensors for interactive machines using user's biological signals (2008)
  • Author(s): T. Tamei
  • Journal Title: Advanced Robotics 22 Pages: 893-911
  • Peer Reviewed
• [Journal Article] Prediction of aperiodic target sequences by saccades (2008)
  • Author(s): M. Shikauchi
  • Journal Title: Behavioural Brain Research 189 Pages: 325-331
  • Peer Reviewed
• [Journal Article] Adaptive control of a looper-like robot based on the CPG-actor-critic method (2008)
  • Author(s): K. Makino
  • Journal Title: Artificial Life and Robotics 12 Pages: 129-132
  • Peer Reviewed
• [Journal Article] A probabilistic modeling of MOSAIC learning (2008)
  • Author(s): S. Osaga
  • Journal Title: Artificial Life and Robotics 12 Pages: 167-171
  • Peer Reviewed
• [Journal Article] Reinforcement learning for a biped robot based on a CPG-actor-critic method (2007)
  • Author(s): Y,Nakamura., T, Mori .,M,Sato., S,Ishii
  • Journal Title: Neural Networks 20 Pages: 723-735
  • NAID: 10019850710
  • Peer Reviewed
• [Journal Article] Model-based reinforcement learning for partially- observable games with sampling-based state estimation (2007)
  • Author(s): H,Fujita., S,Ishii
  • Journal Title: Neural Computation 19(11) Pages: 3051-3087
  • Peer Reviewed
• [Journal Article] Multi-agent reinforcement learning for partially-observable games with sampling-based state estimation (2007)
  • Author(s): H. Fujita
  • Journal Title: Neural Computation 19(11) Pages: 3051-3087
  • Peer Reviewed
• [Journal Article] Reinforcement learning for a biped robot based on a CPG-actor-critic method (2007)
  • Author(s): Y. Nakamura
  • Journal Title: Neural Networks 20 Pages: 723-735
  • NAID: 10019850710
  • Peer Reviewed
• [Journal Article] Reinforcement learning for cooperative actions in a partially observable multiagent system (2007)
  • Author(s): Y. Taniguchi
  • Journal Title: Artificial Neural Networks -ICANN2007, Lecture Notes in Computer Science 4669
  • Peer Reviewed
• [Journal Article] Reinforce- ment learning : machine learning and natural learning (2006)
  • Author(s): S,Ishii., W,Yoshida
  • Journal Title: New Generation Computing 24 Pages: 325-350
  • Peer Reviewed
• [Journal Article] 学習ダイナミクスの制御と脳の物質機構 (2006)
  • Author(s): 銅谷 賢治, 石井 信
  • Journal Title: システム/制御/情報 50(8) Pages: 303-308
• [Journal Article] 重点サンプリング法に基づくNatural Actor-Critic法による効果的なサンプルの再利用 (2006)
  • Author(s): 森 健, 中村 泰, 石井 信
  • Journal Title: 電子情報通信学会論文誌 J89-D(5) Pages: 954-966
  • Peer Reviewed
• [Journal Article] Stochastic resonance with a differential code in feedforward network with intra-layer random connections (2006)
  • Author(s): Y,Sakumura.,S,Ishii
  • Journal Title: Neural Networks 19 Pages: 469-476
  • NAID: 10017601168
  • Peer Reviewed
• [Journal Article] Balancing plasticity and stability of on-line learning based on hierarchical Bayesian adaptation of forgetting factors (2006)
  • Author(s): J,Hirayama., J,Yoshimoto., S,Ishii
  • Journal Title: Neurocomputing 69 Pages: 1954-1961
  • Peer Reviewed
• [Journal Article] Switching particle filters for efficient visual tracking (2006)
  • Author(s): T,Bando., T,Shibata., K,Doya., S,Ishii
  • Journal Title: Journal of Robotics and Autonomous Systems 54 Pages: 873-884
  • Peer Reviewed
• [Journal Article] The role of short-term depression in sustained neural activity in the prefrontal cortex (2006)
  • Author(s): Y,Igarashi., Y,Sakumura., S,Ishii
  • Journal Title: A simulation study. Neural Networks 19(8) Pages: 1137-1152
  • Peer Reviewed
• [Journal Article] 適応的サンプリングによる階層モデル化された対象の効率的状態推定 (2006)
  • Author(s): 坂東 誉司, 柴田 智広, 清水 幹郎, 石井 信
  • Journal Title: システム制御情報学会論文誌 50(10) Pages: 369-377
  • NAID: 10021999928
  • Peer Reviewed
• [Journal Article] Reinforcement learning for quasi-passive dynamic walking of an unstable biped robot (2006)
  • Author(s): K,Hitomi., T,Shibata., Y,Nakamura., S, Ishii
  • Journal Title: Robotics and Autonomous System 54(12) Pages: 982-988
  • Peer Reviewed
• [Journal Article] Reinforcement learning for quasi-passive dynamic walking of an unstable biped robot (2006)
  • Author(s): K.Hitomi
  • Journal Title: Robotics and Autonomous Systems 54(12) Pages: 982-988
• [Journal Article] 適応的サンプリングによる階層モデル化された対象の効率的状態推定 (2006)
  • Author(s): 坂東 誉司
  • Journal Title: システム制御情報学会論文誌 50(10) Pages: 369-377
• [Journal Article] The role of short time depression in sustained neural activity in the prefrontal cortex : A simulation study (2006)
  • Author(s): Y.Igarashi
  • Journal Title: Neural Networks 19(8) Pages: 1137-1152
• [Journal Article] Switching particle filters for efficient visual tracking (2006)
  • Author(s): T.Bando
  • Journal Title: Journal of Robotics and Autonomous Systems 54 Pages: 873-884
• [Journal Article] Balancing plasticity and stability of on-line learning based on hierarchical Bayesian adaptation of forgetting factors (2006)
  • Author(s): J.Hirayama
  • Journal Title: Neurocomputing 69 Pages: 1954-1961
• [Journal Article] Stochastic resonance with a differential code in a cortical model (2006)
  • Author(s): Y.Sakumura
  • Journal Title: Neural Networks 19 Pages: 469-476
• [Journal Article] 重点サンプリング法に基づくNatural Actor-Critic法による効果的なサンプルの再利用 (2006)
  • Author(s): 森 健
  • Journal Title: 電子情報通信学会論文誌 J89-D(5) Pages: 954-966
• [Journal Article] 学習ダイナミクスの制御と脳の物質機構 (2006)
  • Author(s): 銅谷 賢治
  • Journal Title: トステム/制御/情報 50(8) Pages: 303-308
• [Journal Article] Reinforcement learning : machine learning and natural learning (2006)
  • Author(s): S.Ishii
  • Journal Title: New Generation Computing 24 Pages: 325-350
• [Journal Article] Adaptive control of a looper-like robot based on the CPG-actor-critic method. (2006)
  • Author(s): K.Makino
  • Journal Title: International Symposium on Artificial Life and Robotics
• [Journal Article] On-line variational PCA for adaptive visual tracking (2006)
  • Author(s): T.Date
  • Journal Title: International Symposium on Artificial Life and Robotics
• [Journal Article] A probabilistic modeling of MOSAIC learning (2006)
  • Author(s): S.Osaga
  • Journal Title: International Symposium on Artificial Life and Robotics
• [Journal Article] Estimation of source-filter model via acoustical feature extraction by GA-like algorithm (2006)
  • Author(s): M.Ihara
  • Journal Title: International Symposium on Artificial Life and Robotics
• [Journal Article] Fast and Stable Learning of Quasi-Passive Dynamic Walking by an Unstable Biped Robot based on Off-Policy Natural Actor-Critic (2006)
  • Author(s): T.Ueno
  • Journal Title: IEEE/RSJ International Conference on Intelligent Robots and Systems Pages: 5226-5231
• [Journal Article] Feature extraction for decision-theoretic planning in partially observable environments (2006)
  • Author(s): H.Fujita
  • Journal Title: Artificial Neural Networks - ICANN 2006, Lecture Notes in Computation Science 4132
• [Presentation] Online multibody factorization based on Bayesian principal component analysis of Gaussian mixture models (2008)
  • Author(s): K,Hitomi
  • Organizer: In 15^<th> International Conference on Neural Information Processing Lecture Notes in Computer Science 5506
  • Place of Presentation: Auckland
  • Year and Date: 2008-11-27
• [Presentation] Online Multibody Factorization based on Bayesian Principal Component Analysis of Gaussian Mixture Models. (2008)
  • Author(s): Hitomi, K., Bando, T., Fukaya, N., Ikeda, K., Shibata, T.
  • Organizer: 15th International Conference on Neural Information Processing (ICONIP 2008)
  • Place of Presentation: オークランド(ニュージーランド)
  • Year and Date: 2008-11-27
• [Presentation] Interpreting dopamine activities in stochastic reward tasks (2008)
  • Author(s): A,Asahina
  • Organizer: In 15^<th> International Conference on Neural Information Processing
  • Place of Presentation: Auckland
  • Year and Date: 2008-11-26
• [Presentation] Interpreting dopamine activities in stochastic reward tasks (2008)
  • Author(s): A. Asahina
  • Organizer: International Conference on Neural Information Processing
  • Place of Presentation: オークランド(ニュージーランド)
  • Year and Date: 2008-11-26
• [Presentation] Self-organized reinforce -ment learning based on policy gradient in nonstationary environment (2008)
  • Author(s): Y,Hiei
  • Organizer: In Artificial Neural Networks - ICANN 2008, Lecture Notes in Computer Science 5163
  • Place of Presentation: Prague
  • Year and Date: 2008-09-03
• [Presentation] A continuous internal-state controller for partially observable Markov decision processes (2008)
  • Author(s): Y,Taniguchi
  • Organizer: In Artificial Neural Networks- ICANN 2008 Lecture Notes in Computer Science 5163
  • Place of Presentation: Prague
  • Year and Date: 2008-09-03
• [Presentation] A continuous internal-state controller for partially observable Markov decision processes (2008)
  • Author(s): Y. Taniguchi
  • Organizer: Artificial Neural Networks-ICANN 2008, Lecture Notes in Computer Science
  • Place of Presentation: プラハ(チェコ)
  • Year and Date: 2008-09-03
• [Presentation] Self-organized reinforcement learning based on policy gradient in nonstationary environment (2008)
  • Author(s): Y. Hiei
  • Organizer: Artificial Neural Networks-ICANN 2008, Lecture Notes in Computer Science
  • Place of Presentation: プラハ(チェロ)
  • Year and Date: 2008-09-03
• [Presentation] A semiparametric statistical approach to model-free policy evaluation (2008)
  • Author(s): T,Ueno
  • Organizer: In the 25th International Conference on Machine Learning (ICML)
  • Place of Presentation: Helsinki
  • Year and Date: 2008-07-08
• [Presentation] A semiparametric statistical approach to model-free policy evaluation (2008)
  • Author(s): T. Ueno
  • Organizer: The 25th International Conference on Machine Learning (ICML)
  • Place of Presentation: ヘルシンキ(フィンランド)
  • Year and Date: 2008-07-08
• [Presentation] Adaptive particle allo- cation for multifocal visual attention based on particle filtering (2008)
  • Author(s): N,Yano
  • Organizer: In the 13th International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu
  • Year and Date: 2008-01-31
• [Presentation] An effective reinforce -ment learning with automatic construction of basis functions and sequential approximation (2008)
  • Author(s): N,Nanjo
  • Organizer: In the 13th International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu
  • Year and Date: 2008-01-31
• [Presentation] Continuous internal-state controller for a partially observable linear dynamical system (2008)
  • Author(s): Y,Taniguchi
  • Organizer: In the 13th International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu
  • Year and Date: 2008-01-31
• [Presentation] Active sampling based on Gaussian process for reinforcement learning (2008)
  • Author(s): K,Takeda
  • Organizer: In the thirteenth International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu
  • Year and Date: 2008-01-31
• [Presentation] Active sampling based on Gaussian process for reinforcement learning (2008)
  • Author(s): K. Takeda
  • Organizer: Thirteenth International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu, Japan
  • Year and Date: 2008-01-31
• [Presentation] Continuous internal-state controller for a partially observable linear dynamical system (2008)
  • Author(s): Y. Taniguchi
  • Organizer: Thirteenth International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu, Japan
  • Year and Date: 2008-01-31
• [Presentation] An effective reinforcement learning with automatic construction of basis functions and sequential approximation (2008)
  • Author(s): N. Nanjo
  • Organizer: Thirteenth International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu, Japan
  • Year and Date: 2008-01-31
• [Presentation] Adaptive particle allocation for multifocal visual attention based on particle filterinor (2008)
  • Author(s): N. Yano
  • Organizer: Thirteenth International Symposium on Artificial Life and Robotics
  • Place of Presentation: Beppu, Japan
  • Year and Date: 2008-01-31
• [Presentation] Reinforcement learning for cooperative actions in a partially observable multi-agent system (2007)
  • Author(s): Y,Taniguchi
  • Organizer: In Artificial Neural Networks-ICANN 2007 Lecture Notes in Computer Science 4668
  • Place of Presentation: Porto
  • Year and Date: 2007-09-09
• [Presentation] Convergence analysis of the EM algorithm and joint minimization of free energy (2007)
  • Author(s): S,Maeda
  • Organizer: In IEEE Machine Learning for Signal Processing Workshop
  • Year and Date: 2007-08-27
• [Presentation] Convergence analysis of the EM algorithm and joint minimization of freeenergy (2007)
  • Author(s): S. Maeda
  • Organizer: IEEE Machine Learning for Signal Processing Workshop
  • Place of Presentation: Thessaloniki, Greece
  • Year and Date: 2007-08-27
• [Presentation] A probabilistic model of MOSAIC (2007)
  • Author(s): S,Osaga
  • Organizer: In Proceedings of the IEEE Symposium on Foundations of Computational Intelligence
  • Place of Presentation: Hawaii
  • Year and Date: 2007-04-01
• [Presentation] A probabilistic model of MOSAIC (2007)
  • Author(s): S. Osaga
  • Organizer: IEEE Symposium on Foundations of Computational Intelligence
  • Place of Presentation: Hawaii, USA
  • Year and Date: 2007-04-01
• [Presentation] Estimation of source- filter model via acoustical feature extraction by GA-like algorithm (2007)
  • Author(s): M,Ihara
  • Organizer: In the 12th International Symposium on Artificial Life and Robotics
  • Place of Presentation: Oita
  • Year and Date: 2007-01-26
• [Presentation] On-line variational PCA for adaptive visual tracking (2007)
  • Author(s): T,Date
  • Organizer: In the 12th International Symposium on Artificial Life and Robotics
  • Place of Presentation: Oita
  • Year and Date: 2007-01-26
• [Presentation] A probabilistic modeling of MOSAIC learning (2007)
  • Author(s): S,Osaga
  • Organizer: In the 12th International Symposium on Artificial Life and Robotics
  • Place of Presentation: Oita
  • Year and Date: 2007-01-25
• [Presentation] Adaptive control of a looper-like robot based on the CPG -actor-critic method (2007)
  • Author(s): K,Makino
  • Organizer: In the 12th Inter -national Symposium on Artificial Life and Robotics
  • Place of Presentation: Oita
  • Year and Date: 2007-01-25
• [Presentation] Fast and Stable Learning of Quasi-Passive Dynamic Walking by an Unstable Biped Robot based on Off-Policy Natural Actor-Critic (2006)
  • Author(s): T,Ueno
  • Organizer: In Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • Place of Presentation: Beijing
  • Year and Date: 2006-10-30
• [Presentation] Feature extraction for decision-theoretic planning in partially observable environments (2006)
  • Author(s): H,Fujita
  • Organizer: In Artificial Neural Networks-ICANN2006 Lecture Notes in Computer Science 4131
  • Place of Presentation: Athens
  • Year and Date: 2006-09-14
• [Book] A proba -bility primer(eds.,,K, Doya., S,Ishii., A.Pouget., and R.P.N. Rao) (2007)
  • Author(s): Doya, K., & Ishii, S
  • Total Pages: 323
  • Publisher: In Bayesian Brain
• [Book] 統計数理は隠された未来をあらわにする, 分担執筆 (2007)
  • Author(s): 石井 信.
  • Publisher: 東京電機大学出版
• [Book] 統計数理は隠された未来をあらわにする, 分担執筆(2章, pp. 33-55) (2007)
  • Author(s): 石井信
  • Total Pages: 136
  • Publisher: 東京電機大学出版局
• [Book] Bayesian Brain (Chapter 1 : A probability primer) (2006)
  • Author(s): K.Doya
  • Total Pages: 323
  • Publisher: MIT Press
• [Remarks] 論理生命学(石井)研究室HP
  • URL: http://hawaii.sys.i.kyoto-u.ac.jp/home
• [Remarks] 石井信HP
  • URL: http://hawaii.naist.jp/~ishii/index-ja.html
• [Patent(Industrial Property Rights)] 適応型制御器、適応型制御方法および適応型制御プログラム (2007)
  • Inventor(s): 吉本 潤一郎 石井 信
  • Industrial Property Rights Holder: 独立行政法人科学技術振興機構 国立大学法人奈良先端科学技術大学院大学
  • Acquisition Date: 2007-05-25
• [Patent(Industrial Property Rights)] 適応型制御器、適応型制御方法および適応型制御プログラム (2007)
  • Inventor(s): 吉本潤一郎, 石井信
  • Industrial Property Rights Holder: 独立行政法人科学技術振興機構、国立大学法人奈良先端科学技術大学院大学
  • Acquisition Date: 2007-05-25