Elucidation of neural circuit mechanisms of sensory and motor cortices based on the duality of inference and control

• Project Area: Extension and validation of unified theories of prediction and action
• Project/Area Number: 23H04975
• Research Category: Grant-in-Aid for Transformative Research Areas (A)
• Allocation Type: Single-year Grants
• Review Section: Transformative Research Areas, Section (IV)
• Research Institution: Okinawa Institute of Science and Technology Graduate University
• Principal Investigator: 銅谷 賢治 沖縄科学技術大学院大学, 神経計算ユニット, 教授 (80188846)
• Project Period (FY): 2023-04-01 – 2028-03-31
• Project Status: Granted (Fiscal Year 2025)
• Budget Amount: ¥129,610,000 (Direct Cost: ¥99,700,000、Indirect Cost: ¥29,910,000); Fiscal Year 2026: ¥24,570,000 (Direct Cost: ¥18,900,000、Indirect Cost: ¥5,670,000); Fiscal Year 2025: ¥22,750,000 (Direct Cost: ¥17,500,000、Indirect Cost: ¥5,250,000); Fiscal Year 2024: ¥22,750,000 (Direct Cost: ¥17,500,000、Indirect Cost: ¥5,250,000); Fiscal Year 2023: ¥36,790,000 (Direct Cost: ¥28,300,000、Indirect Cost: ¥8,490,000)
• Keywords: ベイズ推定 / 最適制御 / 強化学習 / 大脳皮質神経回路 / カルシウムイメージング

Outline of Research at the Start

大脳皮質は６層の回路構造を持つが、視覚野や体性感覚野などその後半分は主に感覚認知に関わり、運動野、前頭前野などその前半分は主に行動制御に関与する。しかし共通の回路構造がなぜ感覚認知と行動制御という異なる機能を実現できるのかは謎である以前に問題にもされてこなかった。しかし近年、知覚のためベイズ推定と行動のための強化学習が共通の計算手法で実現可能なことが明らかになった。
本研究では、ベイズ推定と強化学習の双対性を手がかりとして、推定と制御が大脳皮質の神経回路でいかに実現可能かについて作業仮説を設定し、計算機シミュレーションとマウスの脳活動計測実験により検証する。

Outline of Annual Research Achievements

大脳皮質は6層の回路構造を持つが、視覚野や体性感覚野などその後半分は主に感覚認知に関わり、運動野、前頭前野などその前半分は主に行動制御に関与する。 しかし共通の回路構造がなぜ感覚認知と行動制御という異なる機能を実現できるのかは謎である以前に問題にもされてこなかった。しかし近年、知覚のためベイ ズ推定と行動のための強化学習が共通の計算手法で実現可能なことが明らかになった。
本研究では、ベイズ推定と強化学習の双対性を手がかりとして、推定と制御が大脳皮質の神経回路でいかに実現可能かについて作業仮説を設定し、計算機シ ミュレーションとマウスの脳活動計測実験により検証する。
今年度は、マウス一次感覚野と一次運動野の多層のニューロンの活動のプリズムレンズを用いた計測に加え、広視野2光子顕微鏡による複数領野の同時記録、多点電極プローブによる大脳皮質と皮質下の同時記録のシステムの導入を行なった。アクティブレバーを用いた行動実験系では、押し引きの方向に応じた抵抗条件の知覚推定と、それに応じた運動制御を統一的にとらえる課題を設計し実験を開始した。

Current Status of Research Progress

2: Research has progressed on the whole more than it was originally planned.

Reason

プリズムレンズを用いたマウス一次感覚野と一次運動野の多層のニューロンの活動計測系に加え、広視野2光子イメージングと、多点電極記録システムの導入が順調に進んでいる。

Strategy for Future Research Activity

今年度確立した行動課題のもとでのベイズ推定と最適制御の理論モデルによるデータ解析とシミュレーションにより、感覚野と運動野の浅層と深層ニューロンの計算機構仮説の検証を進める。

Research Products

• [Int'l Joint Research] Oxford University(英国)
• [Int'l Joint Research] University College London(英国)
• [Journal Article] Optogenetic activation of dorsal raphe serotonin neurons induces brain-wide activation (2024)
  • Author(s): Hamada Hiro Taiyo、Abe Yoshifumi、Takata Norio、Taira Masakazu、Tanaka Kenji F.、Doya Kenji
  • Journal Title: Nature Communications Volume: 15 Issue: 1 Pages: 4152-4152
  • DOI: 10.1038/s41467-024-48489-6
  • Peer Reviewed / Open Access
• [Journal Article] Synergizing habits and goals with variational Bayes (2024)
  • Author(s): Han Dongqi、Doya Kenji、Li Dongsheng、Tani Jun
  • Journal Title: Nature Communications Volume: 15 Issue: 1 Pages: 4461-4461
  • DOI: 10.1038/s41467-024-48577-7
  • Peer Reviewed / Open Access
• [Journal Article] The differential effect of optogenetic serotonergic manipulation on sustained motor actions and waiting for future rewards in mice (2024)
  • Author(s): Taira Masakazu、Miyazaki Kayoko W.、Miyazaki Katsuhiko、Chen Jianning、Okitsu-Sakurayama Shiho、Chaudhary Anupama、Nishio Mika、Miyake Tsukasa、Yamanaka Akihiro、Tanaka Kenji F.、Doya Kenji
  • Journal Title: Frontiers in Neuroscience Volume: 18 Pages: 1433061-1433061
  • DOI: 10.3389/fnins.2024.1433061
  • Peer Reviewed / Open Access
• [Journal Article] Intrinsic Rewards for Exploration Without Harm From Observational Noise: A Simulation Study Based on the Free Energy Principle (2024)
  • Author(s): Tinker Theodore Jerome、Doya Kenji、Tani Jun
  • Journal Title: Neural Computation Volume: 36 Issue: 9 Pages: 1854-1885
  • DOI: 10.1162/neco_a_01690
  • Peer Reviewed / Open Access
• [Journal Article] A whole-brain analysis of functional connectivity and immediate early gene expression reveals functional network shifts after operant learning (2024)
  • Author(s): Kasahara Kazumi、Hikishima Keigo、Nakata Mariko、Tsurugizawa Tomokazu、Higo Noriyuki、Doya Kenji
  • Journal Title: NeuroImage Volume: 299 Pages: 120840-120840
  • DOI: 10.1016/j.neuroimage.2024.120840
  • Peer Reviewed / Open Access
• [Journal Article] Adaptation of the inferior temporal neurons and efficient visual processing (2024)
  • Author(s): Yamane Yukako
  • Journal Title: Frontiers in Behavioral Neuroscience Volume: 18 Pages: 1398874-1398874
  • DOI: 10.3389/fnbeh.2024.1398874
  • Peer Reviewed / Open Access
• [Journal Article] Representation of Natural Contours by a Neural Population in Monkey V4 (2024)
  • Author(s): Machida Itsuki、Shishikura Motofumi、Yamane Yukako、Sakai Ko
  • Journal Title: eNeuro Volume: 11 Issue: 3 Pages: ENEURO.0445-23.2024
  • DOI: 10.1523/eneuro.0445-23.2024
  • Peer Reviewed / Open Access
• [Journal Article] Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks (2023)
  • Author(s): Blackwell Kim T.、Doya Kenji
  • Journal Title: PLOS Computational Biology Volume: 19 Issue: 8 Pages: e1011385-e1011385
  • DOI: 10.1371/journal.pcbi.1011385
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing (2023)
  • Author(s): Yamane Yukako、Ito Junji、Joana Cristian、Fujita Ichiro、Tamura Hiroshi、Maldonado Pedro E.、Doya Kenji、Grun Sonja
  • Journal Title: eNeuro Volume: 10 Issue: 10 Pages: 1-19
  • DOI: 10.1523/eneuro.0086-23.2023
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Neuronal Representation of a Working Memory-Based Decision Strategy in the Motor and Prefrontal Cortico-Basal Ganglia Loops (2023)
  • Author(s): Yoshizawa Tomohiko、Ito Makoto、Doya Kenji
  • Journal Title: eNeuro Volume: 10(6) Issue: 6 Pages: ENEURO.0413-22.2023
  • DOI: 10.1523/eneuro.0413-22.2023
  • Peer Reviewed / Open Access
• [Presentation] Reinforcement Learning, Bayesian Inference and the Digital Brain (2025)
  • Author(s): Kenji Doya
  • Organizer: 2025 5th International Conference on Electrical Engineering and Information Technology (CEEIT 2025)
  • Int'l Joint Research / Invited
• [Presentation] 脳神経統合プログラムのデジタル脳プロジェクト (2025)
  • Author(s): 銅谷賢治
  • Organizer: 第26回日本正常圧水頭症学会
  • Invited
• [Presentation] Encoding of action and action values in the primary motor cortex of mice during a lever-pulling task (2025)
  • Author(s): Naohiro Yamauchi
  • Organizer: NEURO2024
  • Int'l Joint Research
• [Presentation] Data for Brain Reference Architecture of NY24CanonicalCorticalMicrocircuitsInference and NY24CanonicalCorticalMicrocircuitsDecisionMaking (2025)
  • Author(s): Naohiro Yamauchi
  • Organizer: The Second International Whole Brain Architecture Workshop
  • Int'l Joint Research
• [Presentation] A computational approach at mesoscopic scale for studying the interaction between claustrum and cortex (2025)
  • Author(s): Razvan Gamanut
  • Organizer: Mechanism of Brain and Mind Workshop 2025
• [Presentation] fMRI study reveals enhanced top-down and bottom-up processes in schizophrenia (2025)
  • Author(s): Shuhei Hara
  • Organizer: COSYNE 2025
  • Int'l Joint Research
• [Presentation] Embodied Evolution of Intrinsically Motivated Reinforcement Learning (2025)
  • Author(s): Tojoarisoa Rakotoaritina
  • Organizer: OIST Machine Learning Workshop 2025
  • Int'l Joint Research
• [Presentation] Dorsal raphe serotonin neurons encode probability not value of future rewards (2024)
  • Author(s): Katauhiko Miyazaki
  • Organizer: FENS Forum 2024
  • Int'l Joint Research
• [Presentation] Dorsal raphe serotonin neurons encode probability not value of future rewards (2024)
  • Author(s): Kayoko W. Miyazaki
  • Organizer: FENS Forum 2024
  • Int'l Joint Research
• [Presentation] 脳と科学とAI (2024)
  • Author(s): Kenji Doya
  • Organizer: トライボロジー学会2024秋
  • Invited
• [Presentation] デジタル脳プロジェクトのめざすもの (2024)
  • Author(s): Kenji Doya
  • Organizer: 第1回 _本放射線医療技術学術_会（JCRTM2024）
  • Invited
• [Presentation] Brain/MINDS 2.0 and the Digital Brain Project (2024)
  • Author(s): Kenji Doya
  • Organizer: the 17th International Conference on Brain Informatics (BI2024)
  • Int'l Joint Research / Invited
• [Presentation] Evolution, Reinforcement Learning, and Mental Simulation (2024)
  • Author(s): Kenji Doya
  • Organizer: the WS-ICRA-2024 Cognition across species: from nature to robotic application
  • Int'l Joint Research / Invited
• [Presentation] Bayesian inference and model-based reinforcement learning (2024)
  • Author(s): Kenji Doya
  • Organizer: FENS Forum 2024
  • Int'l Joint Research / Invited
• [Presentation] Digital Brain Project of Brain/MINDS 2.0 (2024)
  • Author(s): Kenji Doya
  • Organizer: The 1st International Whole Brain Architecture Workshop
  • Int'l Joint Research / Invited
• [Presentation] Bayesian inference, reinforcement learning, and the cortico-basal ganglia circuits (2024)
  • Author(s): Kenji Doya
  • Organizer: CogSci 2024
  • Int'l Joint Research / Invited
• [Presentation] 脳とAIの強みと弱み (2024)
  • Author(s): 銅谷賢治
  • Organizer: 連続セミナー2024第4回「次世代AIモデルに向けた研究開発動向」
  • Invited
• [Presentation] Cortical micro circuits reference architecture for dynamic Bayesian inference (2024)
  • Author(s): Naohiro Yamauchi
  • Organizer: 1st Digital Brain Workshop
• [Presentation] A computational approach at mesoscopic scale for studying the interaction between claustrum and cortex (2024)
  • Author(s): Razvan Gamanut
  • Organizer: JNNS annual meeting Sapporo
• [Presentation] A bottom-up, mesoscale approach for the study of the claustrum function (2024)
  • Author(s): Razvan Gamanut
  • Organizer: NEST conference - online
  • Int'l Joint Research
• [Presentation] A computational bottom-up, mesoscale approach for the study of the interaction between claustrum and cortex (2024)
  • Author(s): Razvan Gamanut
  • Organizer: Society for Claustrum annual meeting
  • Int'l Joint Research
• [Presentation] A computational bottom-up, mesoscale approach for the study of the interaction between claustrum and cortex (2024)
  • Author(s): Razvan Gamanut
  • Organizer: Society for Neuroscience Meeting 2024
  • Int'l Joint Research
• [Presentation] Reversal learning under 2-week bupropion or escitalopram treatment in mice (2024)
  • Author(s): Terezie Sedlinska
  • Organizer: Society for Neuroscience Meeting 2024
  • Int'l Joint Research
• [Presentation] Evolution of Rewards for Food and Motor Action by Simulating Birth and Death (2024)
  • Author(s): Yuji Kanagawa
  • Organizer: ALIFE 2024
  • Int'l Joint Research
• [Presentation] Multilayer imaging of the mouse motor cortex and modeling by reinforcement learning (2024)
  • Author(s): Naohiro Yamauchi, Kenji Doya
  • Organizer: Mechanism of Brain and Mind Winter Workshop
• [Presentation] Bayesian inference, reinforcement learning, and the cortico-basal ganglia circuit (2023)
  • Author(s): Kenji Doya
  • Organizer: International Joint Conference on Neural Networks
  • Int'l Joint Research / Invited
• [Presentation] Can Robots Find Their Own Reward Functions? (2023)
  • Author(s): Kenji Doya
  • Organizer: IEEE ROSE
  • Int'l Joint Research / Invited
• [Presentation] コネクトームで脳はどこまでわかるのか (2023)
  • Author(s): 銅谷賢治
  • Organizer: 第129回日本解剖学会総会
  • Invited
• [Presentation] Change of sensory-motor network revealed by a wide field calcium imaging of marmoset during learning of forelimb movement task (2023)
  • Author(s): Yukakako Yamane, Teppei Ebina, Akitaka Sasagawa, Shin-ichiro Terada, Masato Uemura, Kenichi Ohki, Masanori Matsuzaki, Kenji Doya
  • Organizer: Neuro2023
• [Presentation] The effect of optogenetic serotonergic activation on sustained motor action in a lever pressing task (2023)
  • Author(s): Jianning Chen, Kenji Doya
  • Organizer: Neuro2023
• [Book] Reinforcement learning. Sun R (ed.), The Cambridge Handbook of Computational Cognitive Sciences, 739 - 766 (2024)
  • Author(s): Kenji Doya
  • Total Pages: 1307
  • Publisher: Cambridge University Press
  • ISBN: 9781108620345
• [Book] Computational cognitive models of reinforcement learning. Sun R (ed.), The Cambridge Handbook of Computational Cognitive Sciences, 739 - 766 (2024)
  • Author(s): Kenji Doya
  • Total Pages: 1307
  • Publisher: Cambridge University Press
  • ISBN: 9781108620345
• [Remarks] Neural Computation Unit
  • URL: https://www.oist.jp/ncu
• [Remarks] Neural Computation Unit
  • URL: https://groups.oist.jp/ncu