• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

2020 Fiscal Year Annual Research Report

Elucidation of the Mathematical Basis and Neural Mechanisms of Multi-layer Representation Learning

Planned Research

Project AreaCorrespondence and Fusion of Artificial Intelligence and Brain Science
Project/Area Number 16H06563
Research InstitutionOkinawa Institute of Science and Technology Graduate University

Principal Investigator

銅谷 賢治  沖縄科学技術大学院大学, 神経計算ユニット, 教授 (80188846)

Project Period (FY) 2016-06-30 – 2021-03-31
Keywordsディープラーニング / 強化学習 / モジュール自己組織化
Outline of Annual Research Achievements

1) 多階層表現学習の数理基盤:ディープラーニングを強化学習に用いる場合に、学習の安定性を保証しながらデータ効率を改善するKL正則化強化学習の数理解析の論文が国際会議NeurIPS 2020の口頭発表として採択された。また、隠れ状態を含む強化学習課題に関して、変分リカレントニューラルネットを用いた新たな学習手法の論文をICLR2020で発表した。
2) 多階層表現学習の神経機構:ナ大脳皮質の神経回路において、ボトムアップの感覚情報とトップダウンの予測情報が統合されるメカニズムを明らかにするため、レバーの微小な動きを識別し操作する行動課題を実行中のマウス大脳皮質の異なる層の神経活動をプリズム内視鏡を用いて同時計測する実験を進め、ニューロン集団のダイナミクスの解析結果をAP-CNNおよびJNNS2020で発表した。
3) 全脳レベルのモジュール自己組織化:セロトニンニューロンの前頭眼窩野(OFC)および内側前頭皮質(mPFC)の神経端末の光遺伝学刺激による報酬待機行動の促進と、その特性を再現するベイズ意思決定モデルの論文がScience Advancesが採択された。
2021年度は、コロナウイルス蔓延によるキャンパス閉鎖等のために遅れていた神経活動計測およびオプトジェネティクス操作実験を行い、論文投稿を行なった。

Research Progress Status

令和2年度が最終年度であるため、記入しない。

Strategy for Future Research Activity

令和2年度が最終年度であるため、記入しない。

  • Research Products

    (60 results)

All 2022 2021 2020 Other

All Int'l Joint Research (2 results) Journal Article (13 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 10 results,  Open Access: 12 results) Presentation (39 results) (of which Int'l Joint Research: 18 results,  Invited: 25 results) Remarks (3 results) Patent(Industrial Property Rights) (2 results) Funded Workshop (1 results)

  • [Int'l Joint Research] University of California Berkeley(米国)

    • Country Name
      U.S.A.
    • Counterpart Institution
      University of California Berkeley
  • [Int'l Joint Research] University College London(英国)

    • Country Name
      UNITED KINGDOM
    • Counterpart Institution
      University College London
  • [Journal Article] A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots2022

    • Author(s)
      Taniguchi T, Yamakawa H, Nagai T, Doya K, Sakagami M, Suzuki M, Nakamura T, Taniguchi A
    • Journal Title

      Neural Networks

      Volume: 150 Pages: 293-312

    • DOI

      10.1016/j.neunet.2022.02.026

    • Peer Reviewed / Open Access
  • [Journal Article] Structural relational inference actor-critic for multi-agent reinforcement learning2021

    • Author(s)
      Zhang X, Liu Y, Xu X, Huang Q, Mao H, Carie, A
    • Journal Title

      Neurocomputing

      Volume: 459 Pages: 383-394

    • DOI

      10.1016/j.neucom.2021.07.014

    • Peer Reviewed / Open Access
  • [Journal Article] Canonical cortical circuits and the duality of Bayesian inference and optimal control2021

    • Author(s)
      Doya K
    • Journal Title

      Current Opinion in Behavioral Sciences

      Volume: 41 Pages: 160-167

    • DOI

      10.1016/j.cobeha.2021.07.003

    • Peer Reviewed / Open Access
  • [Journal Article] Increased serotonin prevents compulsion in addiction2021

    • Author(s)
      Miyazaki K, Miyazaki K W
    • Journal Title

      Science

      Volume: 373 Pages: 1197-1198

    • DOI

      10.1126/science.abl6285

    • Open Access
  • [Journal Article] Forward and inverse reinforcement learning sharing network weights and hyperparameters2021

    • Author(s)
      Uchibe E, Doya K
    • Journal Title

      Neural Networks

      Volume: 144 Pages: 138-153

    • DOI

      10.1016/j.neunet.2021.08.017

    • Open Access
  • [Journal Article] ZFPM1 Necessary for Development of Serotonergic Projections Related to Anxiety and Contextual Fear Learning2021

    • Author(s)
      Taira M, Desforges M
    • Journal Title

      Journal of Neuroscience

      Volume: 41 Pages: 3945-3947

    • DOI

      10.1523/JNEUROSCI.3125-20.2021.

    • Peer Reviewed / Open Access
  • [Journal Article] Serotonergic modulation of cognitive computations2021

    • Author(s)
      Doya K, Miyazaki K W, Miyazaki K
    • Journal Title

      Current Opinion in Behavioral Sciences

      Volume: 38 Pages: 116-123

    • DOI

      10.1016/j.cobeha.2021.02.003

    • Peer Reviewed / Open Access
  • [Journal Article] A biologically constrained spiking neural network model of the primate basal ganglia with overlapping pathways exhibits action selection2021

    • Author(s)
      Girard B, Lienard J, Gutierrez C E, Delord B, Doya K
    • Journal Title

      European Journal of Neuroscience

      Volume: 53 Pages: 2254-2277

    • DOI

      10.1111/ejn.14869

    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Diffusion functional MRI reveals global brain network functional abnormalities driven by targeted local activity in a neuropsychiatric disease mouse model2020

    • Author(s)
      Abe Y, Takata N, Sakai Y, Hamada H T, Hiraoka Y, Aida T, Tanaka K, Denis L B, Doya K, Tanaka K F
    • Journal Title

      Neuroimage

      Volume: 223 Pages: 117318

    • DOI

      10.1016/j.neuroimage.2020.117318

    • Peer Reviewed / Open Access
  • [Journal Article] Optimization and validation of diffusion MRI-based fiber tracking with neural tracer data as a reference2020

    • Author(s)
      Gutierrez C E, Skibbe H, Nakae K, Tsukada H, Lienard J, Watakabe A, Hata J, Reisert M, Woodward A, Yamaguchi Y, Yamamori T, Okano H, Ishii S, Doya K
    • Journal Title

      Scientific reports

      Volume: 10 Pages: 21285

    • DOI

      10.1038/s41598-020-78284-4

    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks2020

    • Author(s)
      Han D, Doya K, Tani J
    • Journal Title

      Neural Networks

      Volume: 129 Pages: 149-162

    • DOI

      10.1016/j.neunet.2020.06.002.

    • Peer Reviewed / Open Access
  • [Journal Article] Serotonergic projections to the orbitofrontal and medial prefrontal cortices differentially modulate waiting for future rewards2020

    • Author(s)
      Miyazaki K, Miyazaki K W, Sivori G, Yamanaka A, Tanaka K F. Doya K
    • Journal Title

      SCIENCE ADVANCES

      Volume: 6 Pages: eabc7246

    • DOI

      10.1126/sciadv.abc7246

    • Peer Reviewed / Open Access
  • [Journal Article] AIと脳神経科学―精神神経疾患へのデータ駆動と理論駆動のアプローチ2020

    • Author(s)
      高橋 英彦, 山下 祐一, 銅谷 賢治
    • Journal Title

      Clinical Neuroscience

      Volume: 38 Pages: 1358-1363

  • [Presentation] Canonical Cortical Circuits and the Duality of Inference and Control2022

    • Author(s)
      Doya K
    • Organizer
      神経研究所発表会 (Online)
    • Invited
  • [Presentation] Embodied agents for survival, reproduction, and prediction2022

    • Author(s)
      Doya K
    • Organizer
      International Workshop on Next Generation Embodied Intelligence
    • Int'l Joint Research / Invited
  • [Presentation] Context-dependent interaction of dopamine and serotonin2022

    • Author(s)
      Doya K
    • Organizer
      COSYNE2022 (Online)
    • Int'l Joint Research / Invited
  • [Presentation] Serotonergic Control of Model-based Decision Making2022

    • Author(s)
      Taira M
    • Organizer
      COSYNE2022
    • Int'l Joint Research
  • [Presentation] Natural and artificial reinforcement learning2021

    • Author(s)
      Doya K
    • Organizer
      NeurIPS2021 (Online)
    • Int'l Joint Research / Invited
  • [Presentation] 進化するスマホロボットは何をめざすか2021

    • Author(s)
      銅谷賢治
    • Organizer
      第38回AI美芸研シンポジウム「コンセプチュアルウイルス」
    • Invited
  • [Presentation] 動物の行動と神経活動を予測する2021

    • Author(s)
      銅谷賢治
    • Organizer
      日本動物実験代替法学会第34回大会
    • Invited
  • [Presentation] 強化学習とベイズ推定の神経回路を探る2021

    • Author(s)
      銅谷賢治
    • Organizer
      中部大学 AI数理データサイエンスセンターキックオフシンポジウム
    • Invited
  • [Presentation] The duality of inference and control as a key to understanding canonical cortical circuits2021

    • Author(s)
      Doya K
    • Organizer
      "RIKEN-OIST symposium ""Kinds of Minds-What is thinking?" (Online)
    • Invited
  • [Presentation] 不確かさ判断の神経機構2021

    • Author(s)
      山根ゆか子、銅谷賢治
    • Organizer
      日本視覚学会2021年夏季大会 (Online)
  • [Presentation] マーモセットカルシウムイメージングデータの因果解析2021

    • Author(s)
      Yamane Y, Ebina T, Sasagawa A, Terada S, Uemura M, Ohki K, Matsuzaki M, Doya K
    • Organizer
      日本神経回路学会 (Online)
  • [Presentation] Investigation of information flow and temporal-spatial organization of neurons across cortical layers from multi-depth two-photon imaging data2021

    • Author(s)
      Li Y, Doya K
    • Organizer
      JNNS2021
  • [Presentation] 脳型人工知能による人間理解と応用2021

    • Author(s)
      銅谷賢治
    • Organizer
      サントリーホールディングス株式会社 研究交流会議 (Online)
  • [Presentation] 人工知能は脳から何を学べば良いのか2021

    • Author(s)
      銅谷賢治
    • Organizer
      【応用脳科学コンソーシアム】2021年度応用脳科学アカデミー (Online)
    • Invited
  • [Presentation] Reinforcement learning, Bayesian inference and the brain2021

    • Author(s)
      Doya K
    • Organizer
      AI and Brain Computation (lecture at summer school) (Online)
    • Invited
  • [Presentation] The duality of control and inference as a clue for cracking the codes of frontal and sensory cortical architectures2021

    • Author(s)
      Doya K
    • Organizer
      CJK Symposium 17, Theoretical Brain (Online)
    • Invited
  • [Presentation] The effect of photoinhibition of dorsal raphe serotonin neurons on stationary waiting and sustained motor actions for future rewards2021

    • Author(s)
      Taira M
    • Organizer
      The 44th annual meeting of the Japan Neuroscience Society
  • [Presentation] Duality of control and inference and the cortical circuit architecture2021

    • Author(s)
      Doya K
    • Organizer
      SBDM 2021
    • Int'l Joint Research / Invited
  • [Presentation] What can we learn from the brain for future AI?2021

    • Author(s)
      Doya K
    • Organizer
      ICLR2021
    • Int'l Joint Research / Invited
  • [Presentation] Neural circuit for mental simulation2021

    • Author(s)
      Doya K
    • Organizer
      China-Japan Expert Symposium on Brain Science
    • Int'l Joint Research / Invited
  • [Presentation] 人工知能は脳から何を学べば良いのか2020

    • Author(s)
      銅谷 賢治
    • Organizer
      応用脳科学アカデミーアドバンスコース「脳とAI」第1回
    • Invited
  • [Presentation] AI and brain science2020

    • Author(s)
      Doya K
    • Organizer
      2020 IEEE CIS Summer School on Emerging Research Trends in Computational Intelligence: Theory and Applications
    • Int'l Joint Research / Invited
  • [Presentation] Communication and Self-organization of Intelligent Agents2020

    • Author(s)
      Doya K
    • Organizer
      NOLTA 2020
    • Int'l Joint Research / Invited
  • [Presentation] Extracting information flow across cortical layers from multi-depth two-photon imaging data2020

    • Author(s)
      Li Y, Doya K
    • Organizer
      第63回自動制御連合講演会
    • Invited
  • [Presentation] 人工知能と脳科学の融合と社会2020

    • Author(s)
      銅谷 賢治
    • Organizer
      第8回神経法学研究会
    • Invited
  • [Presentation] 脳とAIの接点から何を学びうるのか2020

    • Author(s)
      銅谷 賢治
    • Organizer
      第5回全脳アーキテクチャシンポジウム
    • Invited
  • [Presentation] Toward the society of AI agents: what should we learn from the brain and human society2020

    • Author(s)
      Doya K
    • Organizer
      International Symposium on Artificial Intelligence and Brain Science
    • Int'l Joint Research / Invited
  • [Presentation] What can we further learn from the brain for artificial intelligence?2020

    • Author(s)
      Doya K
    • Organizer
      Neurotheory Forum
    • Int'l Joint Research / Invited
  • [Presentation] How to let robots learn, develop, communicate and evolve2020

    • Author(s)
      Doya K
    • Organizer
      Latin American Summer School on Cognitive Robotics (LACORO)
    • Int'l Joint Research / Invited
  • [Presentation] What can we further learn from the brain for artificial intelligence2020

    • Author(s)
      Doya K
    • Organizer
      Neuroscience2020
    • Invited
  • [Presentation] Toward multi-scale brain data assimilation2020

    • Author(s)
      Doya K
    • Organizer
      CNS*2020 Workshop: Machine learning and mechanistic modeling for understanding brain in health and disease
    • Int'l Joint Research / Invited
  • [Presentation] Effects of transcranial direct current stimulation in brain-computer interface2020

    • Author(s)
      Kasahara Ki, Nakamura T, Uehara K, Ogata Y, Doya K, Hanakawa T
    • Organizer
      2021 IEEE 3rd Global Conference on life science and technology (LifeTech 2021)
    • Int'l Joint Research
  • [Presentation] Leverage the average: an analysis of KL regularization in reinforcement learning2020

    • Author(s)
      Vieillard N, Kozuno T, Scherrer B, Pietquin O, Munos R, Geist M
    • Organizer
      Neural Information Processing Systems Online Conference 2020 (NeurIPS 2020)
    • Int'l Joint Research
  • [Presentation] Effects of the neural activity in basal ganglia on the choice behavior in rats2020

    • Author(s)
      Sugiura I, Irei T, Doya K, Kurata K, Miyata R
    • Organizer
      The 30th Annual Conference of Japanese Neural Network Society (JNNS2020)
  • [Presentation] Maximum credibility voting (MCV): An integrative approach for accurate diagnosis of major depressive disorder from clinically readily available data2020

    • Author(s)
      himizu Y, Yoshimoto J, Takamura M, Okada G, Matsumoto T, Fuchikami M, Okada S, Morinobu S, Okamoto Y, Yamawaki S, Doya K
    • Organizer
      APSIPA 2020
    • Int'l Joint Research
  • [Presentation] Neuron hubs distributed differently in deep layers and superficial layers in different brain states2020

    • Author(s)
      Li Y, Doya K
    • Organizer
      The 1st Asia-Pacific Computational and Cognitive Neuroscience Conference. (2020 AP-CCN)
    • Int'l Joint Research
  • [Presentation] Investigation of temporal and spatial origination of neural network in sensory cortex.2020

    • Author(s)
      Li Y, Doya K
    • Organizer
      The 30th Annual Conference of Japanese Neural Network Society (JNNS2020)
  • [Presentation] Changes in the basal ganglia-thalamic functional connectivity induced by longitudinal motor training in mice2020

    • Author(s)
      Hikishima-Kasahara K, Doya K
    • Organizer
      Neuroscience 2020
  • [Presentation] Variational recurrent models for solving partially observable control tasks2020

    • Author(s)
      Han D, Doya K, Tani J
    • Organizer
      Eighth International Conference on Learning Representations (ICLR 2020)
    • Int'l Joint Research
  • [Remarks] 沖縄科学技術大学院大学 神経計算ユニット

    • URL

      https://groups.oist.jp/ja/ncu

  • [Remarks] 人工知能と脳科学の対照と融合

    • URL

      http://www.brain-ai.jp/jp/

  • [Remarks] International Symposium on AI and Brain Science

    • URL

      http://www.brain-ai.jp/jp/symposium2020/

  • [Patent(Industrial Property Rights)] Inverse reinforcement learning by density ratio estimation2021

    • Inventor(s)
      Uchibe E, Doya K
    • Industrial Property Rights Holder
      Uchibe E, Doya K
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      10896382
  • [Patent(Industrial Property Rights)] Direct inverse reinforcement learning with density ratio estimation2021

    • Inventor(s)
      Uchibe E, Doya K
    • Industrial Property Rights Holder
      Uchibe E, Doya K
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      10896383
  • [Funded Workshop] International Symposium on Artificial Intelligence and Brain Science 20202020

URL: 

Published: 2022-12-28  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi