• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

2019 Fiscal Year Annual Research Report

Elucidation of the Mathematical Basis and Neural Mechanisms of Multi-layer Representation Learning

Planned Research

Project AreaCorrespondence and Fusion of Artificial Intelligence and Brain Science
Project/Area Number 16H06563
Research InstitutionOkinawa Institute of Science and Technology Graduate University

Principal Investigator

銅谷 賢治  沖縄科学技術大学院大学, 神経計算ユニット, 教授 (80188846)

Project Period (FY) 2016-06-30 – 2021-03-31
Keywordsディープラーニング / 強化学習 / モジュール自己組織化
Outline of Annual Research Achievements

1) 多階層表現学習の数理基盤:ディープラーニングを強化学習に用いる場合に、学習の安定性を保証しながらデータ効率を改善するConservative Value Iterationアルゴリズムの収束速度の数理解析とシミュレーション実験結果を、国際会議AISTATS2019で発表した。また、隠れ状態を含む強化学習課題において、リカレントニューラルネットを用いた新たな学習手法を複数提案し、NeurIPS2019ワークショップで発表、ICLR2020に採択された。
2) 多階層表現学習の神経機構:ナビゲーション課題中のマウスの頭頂葉ニューロンの2光子顕微鏡による活動記録データ(Funamizu et al. 2016)に対して、Demixed PCAなどの新たな解析手法を適用することにより、刺激の有無、ゴールへの距離などの情報が、従来のデコーディング手法ではなく、教師なし学習の手法で抽出できることを明らかにした。
3) 全脳レベルのモジュール自己組織化:大脳皮質の神経回路において、ボトムアップの感覚情報とトップダウンの予測情報が統合されるメカニズムを明らかにするため、レバーの微小な動きを識別し操作する行動課題を実行中のマウス大脳皮質の異なる層の神経活動をプリズム内視鏡を用いて同時計測する実験を進め、多数のニューロンの感覚と予測誤差情報の表現の解析を行った。セロトニンニューロンの神経端末の光遺伝学刺激にり、報酬待機行動の促進には前頭眼窩野(OFC)と内側前頭皮質(mPFC)が関わることろ明らかにし、それらの特性を再現するベイズ意思決定モデルを提案し、論文投稿を行った。

Current Status of Research Progress
Current Status of Research Progress

2: Research has progressed on the whole more than it was originally planned.

Reason

新たな実験や論文投稿が進んだのに加え、確率モデルの安定で効率的な学習アルゴリズムTotal Propagation法の特許出願をもとに、そのオープンソースライブラリとしての公開によるビジネス応用に向けた展開が、OIST Proof Of Conceptプログラムに採択されスタートした。

Strategy for Future Research Activity

1) 多階層表現学習の数理基盤:提案した新たな学習アルゴリズムが安定に効率よく動作することを、ロボット制御などの課題で実証する。
2) 多階層表現学習の神経機構:大脳皮質と大脳基底核の多階層情報表現に関する知見を論文として発表する。
3) 全脳レベルのモジュール自己組織化:大脳皮質でのボトムアップの感覚情報とトップダウンの予測情報が統合されるメカニズムについての光学神経活動計測実験をさらに進める。

  • Research Products

    (35 results)

All 2020 2019 Other

All Journal Article (5 results) (of which Open Access: 4 results,  Peer Reviewed: 2 results) Presentation (28 results) (of which Int'l Joint Research: 18 results,  Invited: 18 results) Remarks (2 results)

  • [Journal Article] Gap-Increasing Policy Evaluation for Efficient and Noise-Tolerant Reinforcement Learning.2019

    • Author(s)
      Kozuno T, Han D, Doya K
    • Journal Title

      arXiv.

      Volume: N/A Pages: 1906.07586.

    • DOI

      https://arxiv.org/abs/1906.07586

    • Open Access
  • [Journal Article] A unified view of likelihood ratio and reparameterization gradients and an optimal importance sampling scheme.2019

    • Author(s)
      Parmas P, Sugiyama M
    • Journal Title

      arXiv.

      Volume: N/A Pages: 1910.06419

    • DOI

      https://arxiv.org/abs/1910.06419

    • Open Access
  • [Journal Article] Toward evolutionary and developmental intelligence2019

    • Author(s)
      Doya K, Taniguchi T
    • Journal Title

      Current Opinion in Behavioral Sciences

      Volume: 29 Pages: 91-96

    • DOI

      10.1016/j.cobeha.2019.04.006

    • Peer Reviewed / Open Access
  • [Journal Article] 人工知能と脳科学の現在とこれから2019

    • Author(s)
      銅谷賢治、松尾豊
    • Journal Title

      BRAIN and NERVE 増大特集 人工知能と神経科学

      Volume: 71(7) Pages: 649-655

    • DOI

      10.11477/mf.1416201337.

  • [Journal Article] Theoretical analysis of efficiency and robustness of softmax and gap-increasing operators in reinforcement learning2019

    • Author(s)
      Kozuno T, Uchibe E, Doya K
    • Journal Title

      Proceedings of Machine Learning Research (PMLR)

      Volume: 89 Pages: 2995-3003

    • DOI

      http://proceedings.mlr.press/v89/kozuno19a.html

    • Peer Reviewed / Open Access
  • [Presentation] How Can the Brain Connect Predictors and Actors on the Fly?2020

    • Author(s)
      Doya K
    • Organizer
      Workshop on Learning for flexible, context-sensitive behavior
    • Int'l Joint Research / Invited
  • [Presentation] Identifying the Evolutionary Conditions for the Emergence of Alternative Reproductive Tactics in Simulated Robot Colonies2020

    • Author(s)
      Rahman F, Doya K, Mickheyev A
    • Organizer
      AROB 25th 2020
    • Int'l Joint Research
  • [Presentation] What can we further learn from the brain for artificial intelligence?2020

    • Author(s)
      Doya K
    • Organizer
      脳と心のメカニズム第20回冬のワークショップ
    • Int'l Joint Research / Invited
  • [Presentation] Representation and grounding of abstract concepts: a preliminary investigation. Mechanism of Mind and Brain2020

    • Author(s)
      Chiu HC, Doya K
    • Organizer
      脳と心のメカニズム第20回冬のワークショップ
  • [Presentation] セロトニンによる報酬待機行動の制御機構2019

    • Author(s)
      宮崎勝彦
    • Organizer
      第254回つくばブレインサイエンス・セミナー
    • Invited
  • [Presentation] What Can We Learn from the Brain for Next AI?2019

    • Author(s)
      Doya K
    • Organizer
      The 3rd Ryudai-OIST Symposium: Basic Medical Science to Clinical Medicine
  • [Presentation] Intrinsic motivation in creative activity: A human behavioral experiment for identifying the factors that influence intrinsic motivation.2019

    • Author(s)
      Ota S, Doya K
    • Organizer
      WIML, NeurIPS
    • Int'l Joint Research
  • [Presentation] A unified view of likelihood ratio and reparameterization gradients and an optimal importance sampling scheme2019

    • Author(s)
      Parmas P, Sugiyama M
    • Organizer
      Deep Reinforcement Learning Workshop, NeurIPS
    • Int'l Joint Research
  • [Presentation] マウスの脳内シミュレーションとロボットの報酬進化2019

    • Author(s)
      銅谷賢治
    • Organizer
      第12回日本人間行動進化学会
    • Invited
  • [Presentation] Consciousness as Data Assimilation2019

    • Author(s)
      Doya K
    • Organizer
      Consciousness Club. ARAYA
    • Invited
  • [Presentation] Toward whole-brain multi-scale modeling2019

    • Author(s)
      Doya K
    • Organizer
      NBNI: Neurobiology and Neuroinformatics
    • Int'l Joint Research
  • [Presentation] Patience, Confidence and Serotonin2019

    • Author(s)
      Doya K
    • Organizer
      Blue Brain Seminar
    • Invited
  • [Presentation] What Can We Further Learn from the Brain for Cognitive Robotics?2019

    • Author(s)
      Doya K
    • Organizer
      IROS 2019 Workshop: Deep Probabilistic Generative Models for Cognitive Architecture in Robotics
    • Int'l Joint Research / Invited
  • [Presentation] Systems Biology of Reinforcement Learning2019

    • Author(s)
      Doya K
    • Organizer
      International Conference on Systems Biology (ICSB 2019)
    • Int'l Joint Research
  • [Presentation] Reinforcement Learning in Machines and the Brain2019

    • Author(s)
      Doya K
    • Organizer
      Conference on Robot Learning (CoRL 2019)
    • Int'l Joint Research / Invited
  • [Presentation] Stimulation of serotonergic terminals in the orbitofrontal and medial prefrontal cortices differentially affects waiting for the future rewards2019

    • Author(s)
      Miyazaki KW, Miyazaki K, Yamanaka A, Tanaka KF, Doya K
    • Organizer
      Society for Neuroscience (SfN 2019)
    • Int'l Joint Research
  • [Presentation] Artificial Intelligence and Brain Science2019

    • Author(s)
      Doya K
    • Organizer
      The Joint Symposium of WPI-IIIS, Ph.D. Program in Humanics, and 36th Takamine Conference
    • Invited
  • [Presentation] Neural Circuits for Reinforcement Learning and Mental Simulation2019

    • Author(s)
      Doya K
    • Organizer
      IBRO 2019 Symposium: Valence and Reward Encoding
    • Int'l Joint Research / Invited
  • [Presentation] ベイジアンブレインの今日2019

    • Author(s)
      銅谷賢治
    • Organizer
      日本認知科学会第36回大会
    • Invited
  • [Presentation] Intrinsic motivation in play: Preliminary experiment for analyzing how learning environmental condition influences intrinsic motivation2019

    • Author(s)
      Ota S, Doya K
    • Organizer
      International Conference on Cognitive Science (ICCS 2019)
    • Int'l Joint Research
  • [Presentation] Neural circuits for mental simulation2019

    • Author(s)
      Doya K
    • Organizer
      International Conference on Cognitive Science (ICCS2019)
    • Int'l Joint Research / Invited
  • [Presentation] What can we further learn from the brain for AI and robotics?2019

    • Author(s)
      Doya K
    • Organizer
      Global AI Summit: Cognitive AI
    • Int'l Joint Research / Invited
  • [Presentation] Big Data Challenges in Neuroscience2019

    • Author(s)
      Doya K
    • Organizer
      IEEE CIS Summer School: Big Data Analytics and Stream Processing
    • Int'l Joint Research / Invited
  • [Presentation] Neural Circuits for Mental Simulation2019

    • Author(s)
      Doya K
    • Organizer
      CNS*2019, 28th Annual Computational Neuroscience Meeting
    • Int'l Joint Research / Invited
  • [Presentation] Possible roles of dopamine in model-free and model-based decision and learning2019

    • Author(s)
      Doya K
    • Organizer
      CNS*2019, Dopaminergic Signaling Workshop
    • Int'l Joint Research / Invited
  • [Presentation] Ethologically Grounded Motivation and Neural Implementation of Mental Simulation2019

    • Author(s)
      Doya K
    • Organizer
      The 4th International Workshop on Intrinsically Motivated Open-ended Learning (IMOL2019)
    • Int'l Joint Research / Invited
  • [Presentation] 人工知能と脳科学2019

    • Author(s)
      銅谷賢治
    • Organizer
      第58回日本生体医工学会大会
    • Invited
  • [Presentation] Learning to Communicate for Ecological Fitness2019

    • Author(s)
      Doya K
    • Organizer
      Shonan Meeting No. 141. Language as Goal-Directed Sequential Behavior: Computational Theories, Brain Mechanisms, Evolutionary Roots
  • [Remarks] 人工知能と脳科学の対照と融合

    • URL

      http://www.brain-ai.jp/jp/

  • [Remarks] 沖縄科学技術大学院大学 神経計算ユニット

    • URL

      https://groups.oist.jp/ncu

URL: 

Published: 2021-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi