予測と意思決定の神経回路ダイナミクスの解明

2015 Fiscal Year Annual Research Report

• Project Area: Elucidation of neural computation for prediction and decision making: toward better human understanding and applications
• Project/Area Number: 23120007
• Research Institution: Okinawa Institute of Science and Technology Graduate University
• Principal Investigator: 銅谷 賢治 沖縄科学技術大学院大学, 神経計算ユニット, 教授 (80188846)
• Project Period (FY): 2011-04-01 – 2016-03-31
• Keywords: 意思決定 / モデルフリー / モデルベース / 強化学習 / ベイズ推定 / 二光子顕微鏡 / 大脳基底核 / 大脳皮質

Outline of Annual Research Achievements

1)脳内シミュレーション過程の可視化：
前進すると音源位置が接近し、音源に最接近した時点でリッキングを行うと水報酬が得られるという「聴覚仮想ナビゲーション課題」を実行中のマウスの、頭頂連合野および第２次視覚野のニューロン活動の二光子顕微鏡による記録実験とデータ解析を行った。
ニューロン活動データから音源位置を推定するベイズデコーダーにより、音が提示されない期間にも音源位置の予測が更新され、音亭提示によりその不確かさが減少するという、動的ベイズフィルターを特徴づける振る舞いが確認された。この結果を論文にまとめ投稿した。
2)モデルベースとモデルフリーの意思決定の神経回路：
確率報酬による意思決定課題中のラットの行動系列は有限状態マシンとしてモデル化し予測できること、その特定の状態遷移に関わるニューロンが背内側線条体に存在することを明らかにし、論文を出版した。
内視鏡型超小型顕微鏡による海馬および線条体の神経活動の光学記録とデータ解析を行い、海馬には場所だけでなく歩行の停止に関連する活動成分が強く見られること、線条体のストライオソームでは報酬予測に応じた活動が条件付学習にともない形成されることを明らかにした。

Research Progress Status

27年度が最終年度であるため、記入しない。

Strategy for Future Research Activity

27年度が最終年度であるため、記入しない。

Research Products

• [Journal Article] Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex (2016)
  • Author(s): Caligiore D, Pezzulo G, Baldassarre G, Bostan A C., Strick P L., Doya K, Helmich R C., Dirkx M, Houk J, Jörntell H, Lago-Rodriguez A, Galea J M., Miall R. C, Popa T, Kishore A, Verschure P F.M.J., Zucca R,Herreros I
  • Journal Title: The Cerebellum Volume: 15 Pages: 1-27
  • DOI: 10.1007/s12311-016-0763-3
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] 線形可解マルコフ決定過程を用いた順・逆強化学習 (2016)
  • Author(s): 内部英治
  • Journal Title: 日本神経回路学会誌 Volume: 23 Pages: 2-13
  • DOI: 10.3902/jnns.23.2
  • Acknowledgement Compliant
• [Journal Article] EM-based policy hyper parameter exploration: application to standing and balancing of a two-wheeled smartphone robot (2016)
  • Author(s): Wang J, Uchibe E, Doya K
  • Journal Title: Artificial Life and Robotics Volume: 21 Pages: 125-131
  • DOI: 10.1007/s10015-015-0260-7
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Journal Article] Hierarchical control of goal-directed action in the cortical–basal ganglia network (2015)
  • Author(s): Balleine B, Dezfouli A, Ito M, Doya K
  • Journal Title: Current Opinion in Behavioral Sciences Volume: 5 Pages: 1-7
  • DOI: 10.1016/j.cobeha.2015.06.001
  • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
• [Journal Article] Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum (2015)
  • Author(s): Ito M, Doya K
  • Journal Title: PLOS Computational Biology Volume: 11 Pages: e1104540
  • DOI: 10.1371/ journal.pcbi.1004540
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Presentation] Neural Circuits and Neuromodulations Prediction and Decision under Uncertainty:Roles of the Cortical Circuit and Serotonin (2015)
  • Author(s): Doya K
  • Organizer: 7th International Symposium on Optogenetics:Neural Circuits and Neuromodulations
  • Place of Presentation: Akio Suzuki Memorial Hall, Medical and Dental University,Tokyo
  • Year and Date: 2015-12-04 – 2015-12-05
  • Int'l Joint Research / Invited
• [Presentation] モデルベース予測の脳機構 (2015)
  • Author(s): 銅谷賢治
  • Organizer: 日本心理学会7第9回大会
  • Place of Presentation: 名古屋国際会議場
  • Year and Date: 2015-09-23 – 2015-09-23
  • Invited
• [Presentation] Investigating neural implementation of model-based decision making by two-photon microscopy (2015)
  • Author(s): Funamizu A, Kuhn B, Doya K
  • Organizer: The 6th FAONS Congress and 11th Biennial Conference of the Chinese Neuroscience Society (CNS)
  • Place of Presentation: Wuzhen, CHINA
  • Year and Date: 2015-09-22 – 2015-09-22
  • Int'l Joint Research / Invited
• [Presentation] Uncovering representations and algorithms of decision making by model-based analysis of striatal neuron activities (2015)
  • Author(s): Ito M
  • Organizer: The 6th FAONS Congress and 11th Biennial Conference of the Chinese Neuroscience Society (CNS)
  • Place of Presentation: Wuzhen, CHINA
  • Year and Date: 2015-09-22 – 2015-09-22
  • Int'l Joint Research / Invited
• [Presentation] Neuronal Mechanism of Reward-Based Learning and Decision (2015)
  • Author(s): Doya K
  • Organizer: Brain Conference 2015; Joint Conference of KSBNS and KSND
  • Place of Presentation: Korea Brain Research Institute, Daegu, KOREA
  • Year and Date: 2015-09-11 – 2015-09-11
  • Int'l Joint Research / Invited
• [Presentation] 行動学習の多層的な表現・アルゴリズムとその制御 (2015)
  • Author(s): 銅谷賢治
  • Organizer: 日本発達神経科学学会第４回学術集会
  • Place of Presentation: 大阪大学
  • Year and Date: 2015-09-06 – 2015-09-06
  • Invited
• [Presentation] Serotonin and the regulation of patience (2015)
  • Author(s): Doya K, Miyazaki Kayoko W, Miyazaki Katsuhiko
  • Organizer: Neuroscience2015-38th Annual Meeting of the Japan Neuroscience Society
  • Place of Presentation: Kobe International Conference Center
  • Year and Date: 2015-07-30 – 2015-07-30
  • Int'l Joint Research / Invited
• [Presentation] Action-dependent state prediction in mouse parietal cortex during an auditory virtual navigation task (2015)
  • Author(s): Funamizu A, Kuhn B, Doya K
  • Organizer: Neuroscience2015-38th Annual Meeting of the Japan Neuroscience Society
  • Place of Presentation: Kobe International Conference Center
  • Year and Date: 2015-07-29 – 2015-07-29
  • Int'l Joint Research
• [Presentation] The role of the cortico-basal ganglia loops for reward-motivated choice motor behavior (2015)
  • Author(s): Yoshizawa T, Ito M, Doya K
  • Organizer: Neuroscience2015- 38th Annual Meeting of the Japan Neuroscience Society
  • Place of Presentation: Kobe International Conference Center
  • Year and Date: 2015-07-29 – 2015-07-29
  • Int'l Joint Research
• [Presentation] Neural representations of memory-based and value-based decision strategies in the cortico-basal ganglia loops (2015)
  • Author(s): Ito M, Doya K
  • Organizer: Neuroscience 2015-38th Annual Meeting of the Japan Neuroscience Society
  • Place of Presentation: Kobe International Conference Center
  • Year and Date: 2015-07-28 – 2015-07-28
  • Int'l Joint Research
• [Presentation] How do we define the rewards in reinforcement learning? (2015)
  • Author(s): Uchibe E
  • Organizer: 電子情報通信学会ニューロコンピューティング研究会
  • Place of Presentation: OIST
  • Year and Date: 2015-06-23 – 2015-06-25
  • Invited
• [Presentation] Inverse Reinforcement Learning with Density Ratio Estimation (2015)
  • Author(s): Uchibe E, Doya K
  • Organizer: The Multi-disciplinary Conference on Reinforcement Learning and Decision Making 2015(RLDM2015)
  • Place of Presentation: University of Alberta, Edmonton, Canada
  • Year and Date: 2015-06-08 – 2015-06-08
  • Int'l Joint Research
• [Presentation] Neural implementation of mental simulation (2015)
  • Author(s): Doya K
  • Organizer: The 5th International conference on Cognitive Neurodynamics
  • Place of Presentation: Sanya, China
  • Year and Date: 2015-06-05 – 2015-06-05
  • Int'l Joint Research / Invited
• [Presentation] 後頭頂葉におけるモデルベース意思決定時の神経表現 (2015)
  • Author(s): 船水章大
  • Organizer: 第54回日本生体医工学会大会
  • Place of Presentation: 名古屋国際会議場
  • Year and Date: 2015-05-07 – 2015-05-07
  • Invited
• [Remarks] 「予測と意思決定」新学術領域webページ
  • URL: http://www.decisions.jp
• [Remarks] 沖縄科学技術大学院大学 神経計算ユニット
  • URL: https://groups.oist.jp/ncu
• [Funded Workshop] International Symposium on Prediction and Decision Making 2015 (2015)
  • Place of Presentation: Koshiba Hall, The University of Tokyo
  • Year and Date: 2015-10-31 – 2015-11-01