• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Identification of neural basis of animal behavioral strategy by inverse reinforcement learning

Research Project

Project/Area Number 16K16147
Research Category

Grant-in-Aid for Young Scientists (B)

Allocation TypeMulti-year Fund
Research Field Life / Health / Medical informatics
Research InstitutionKyoto University

Principal Investigator

Honda Naoki  京都大学, 生命科学研究科, 准教授 (30515581)

Project Period (FY) 2016-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2019: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2018: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2017: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2016: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Keywords逆強化学習 / 機械学習 / 行動戦略 / 意志決定 / 線虫 / 報酬 / データ解析 / 数里モデル / 数理モデル / 人工知能 / 行動学 / 神経科学 / 制御工学 / 動物
Outline of Final Research Achievements

Animals adapt to the environments by acquiring behavioral strategies, which maximize cumulative rewards. However, method to quantify the behavioral strategies of free-living animals has not been fully established. Here, we developed the inverse reinforcement learning method to estimate the behavioral strategy as a value function from behavioral time-series data. By applying this method to the thermotaxis behavior of C. elegans, we successfully identified the behavioral strategy. Our method provides a fundamental technique for linking neural activity and its phenotypic behavioral strategies and can be expected to make a significant contribution to the elucidation of the neural mechanisms for the behavioral strategies.

Academic Significance and Societal Importance of the Research Achievements

ヒトや動物は、より多くの報酬を得るため、状況に応じた「行動戦略」を持って生きている。報酬には食べ物やお金など実態の伴うものだけでなく、間接的にそれらに結びつくものも含まれており、自由に行動している動物を単に観察しているだけでは、「動物にとって何が報酬となっているのか?」を知ることは困難であった。そこで本研究では動物の行動データから報酬に基づく行動戦略を明らかにする機械学習法を考案した。そして、線虫の行動へと適用することで、手法の有効性を示した。本手法によって、従来の行動が制限された行動実験系から開放され、より自然な状況おいて自由に振る舞う動物の行動戦略の研究が進むことが期待される。

Report

(5 results)
  • 2019 Annual Research Report   Final Research Report ( PDF )
  • 2018 Research-status Report
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (31 results)

All 2019 2018 2017 2016 Other

All Journal Article (13 results) (of which Int'l Joint Research: 10 results,  Peer Reviewed: 11 results,  Open Access: 10 results,  Acknowledgement Compliant: 2 results) Presentation (15 results) (of which Int'l Joint Research: 7 results,  Invited: 6 results) Book (2 results) Remarks (1 results)

  • [Journal Article] Noise-resistant developmental reproducibility in vertebrate somite formation2019

    • Author(s)
      Honda N*, Akiyama R, Sari DWK, Ishii S, Bessho Y and Matsui T
    • Journal Title

      PLoS Computational Biology

      Volume: 15(2) Issue: 2 Pages: e1006579-e1006579

    • DOI

      10.1371/journal.pcbi.1006579

    • NAID

      120006553009

    • Related Report
      2018 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Identification of animal behavioral strategies by inverse reinforcement learning.2018

    • Author(s)
      Yamaguchi S, Honda N*, Ikeda M, Tsukada Y, Nakano S, Mori I and Ishii S
    • Journal Title

      PLoS Computational Biology

      Volume: 14(5) Issue: 5 Pages: e1006122-e1006122

    • DOI

      10.1371/journal.pcbi.1006122

    • NAID

      120006469168

    • Related Report
      2018 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Time-lapse observation of stepwise regression of Erk activity in zebrafish presomitic mesoderm.2018

    • Author(s)
      Sari DWK, Akiyama R, Naoki H, Ishijima H, Bessho Y, Matsui T
    • Journal Title

      Scientific Reports

      Volume: 8 Issue: 1 Pages: 4335-4335

    • DOI

      10.1038/s41598-018-22619-9

    • NAID

      120006555360

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Identification of animal behavioral strategies by inverse reinforcement learning2017

    • Author(s)
      Yamaguchi S, Honda N, Ikeda M, Tsukada Y, Nakano S, Mori I and Ishii S
    • Journal Title

      bioRxiv

      Volume: 129007 Pages: 129007-129007

    • DOI

      10.1101/129007

    • NAID

      120006469168

    • Related Report
      2017 Research-status Report
    • Open Access / Int'l Joint Research
  • [Journal Article] Discovery of long-range inhibitory signaling to ensure single axon formation2017

    • Author(s)
      Takano Tetsuya、Wu Mengya、Nakamuta Shinichi、Naoki Honda、Ishizawa Naruki、Namba Takashi、Watanabe Takashi、Xu Chundi、Hamaguchi Tomonari、Yura Yoshimitsu、Amano Mutsuki、Hahn Klaus M.、Kaibuchi Kozo
    • Journal Title

      Nature Communications

      Volume: 8 Issue: 1 Pages: 33-33

    • DOI

      10.1038/s41467-017-00044-2

    • NAID

      120006325188

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Revisiting chemoaffinity theory: Chemotactic implementation of topographic axonal projection2017

    • Author(s)
      Honda Naoki
    • Journal Title

      PLoS Computational Biology

      Volume: 13 Issue: 8 Pages: 8-8

    • DOI

      10.1371/journal.pcbi.1005702

    • NAID

      120006360789

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Propagating Wave of ERK Activation Orients Collective Cell Migration2017

    • Author(s)
      Aoki Kazuhiro、Kondo Yohei、Naoki Honda、Hiratsuka Toru、Itoh Reina E.、Matsuda Michiyuki
    • Journal Title

      Developmental Cell

      Volume: 43 Issue: 3 Pages: 305-317.e5

    • DOI

      10.1016/j.devcel.2017.10.016

    • Related Report
      2017 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Nonrandom contribution of left and right testes to germline transmission from mouse spermatogonial stem cells2017

    • Author(s)
      Mito Kanatsu-Shinohara, Honda Naoki, Takashi Shinohara
    • Journal Title

      Biology of Reproduction

      Volume: 97(6) Issue: 6 Pages: 902-910

    • DOI

      10.1093/biolre/iox141

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Self-organization mechanism of distinct microtubule orientations in axons and dendrites2017

    • Author(s)
      Honda Naoki, Katsuhiro Uegaki, Shin Ishii
    • Journal Title

      bioRxiv

      Volume: 163014 Pages: 1-18

    • DOI

      10.1101/163014

    • Related Report
      2017 Research-status Report
    • Open Access
  • [Journal Article] Nonrandom germline transmission of mouse spermatogonial stem cells.2016

    • Author(s)
      Kanatsu-Shinohara M., Naoki H., Shinohara T.
    • Journal Title

      Dev Cell

      Volume: 38 (3) Issue: 3 Pages: 248-261

    • DOI

      10.1016/j.devcel.2016.07.011

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Int'l Joint Research / Acknowledgement Compliant
  • [Journal Article] Uncertainty-Dependent Extinction of Fear Memory in an Amygdala-mPFC Neural Circuit Model2016

    • Author(s)
      Yuzhe Li, Ken Nakae, Shin Ishii, Honda Naoki
    • Journal Title

      PLoS Comput Biol

      Volume: 12 (9) Issue: 9 Pages: e1005099-e1005099

    • DOI

      10.1371/journal.pcbi.1005099

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research / Acknowledgement Compliant
  • [Journal Article] Two New FRET Imaging Measures: Linearly Proportional to and Highly Contrasting the Fraction of Active Molecules.2016

    • Author(s)
      Yamao M, Aoki K, Yukinawa N, Ishii S, Matsuda M, Naoki H.
    • Journal Title

      PLoS One

      Volume: 11 Issue: 10 Pages: e0164254-e0164254

    • DOI

      10.1371/journal.pone.0164254

    • NAID

      120005853281

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Multi-phasic bi-directional chemotactic responses of the growth cone.2016

    • Author(s)
      Naoki H, Nishiyama M, Togashi K, Igarashi Y, Hong K and Ishii S
    • Journal Title

      Scientific Reports

      Volume: 6 Issue: 1 Pages: 36256-36256

    • DOI

      10.1038/srep36256

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] Deciphering animal behavioral strategy by inverse reinforcement learning2019

    • Author(s)
      Honda Naoki
    • Organizer
      2019 A3 Workshop on Mathematical Life Science
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] 逆強化学習による動物行動戦略の解読2019

    • Author(s)
      本田直樹
    • Organizer
      生理研研究会2019 認知神経科学の先端「脳の理論から身体・世界へ」
    • Related Report
      2019 Annual Research Report
  • [Presentation] 逆強化学習による「動物の行動戦略」のデータ駆動的解読2019

    • Author(s)
      本田直樹
    • Organizer
      生物工学会
    • Related Report
      2019 Annual Research Report
  • [Presentation] Identification of animal behavioral strategy by inverse reinforcement learning2019

    • Author(s)
      Honda Naoki
    • Organizer
      International Conference of Systems Biology
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Identification of animal behavioral and learning principles by inverse reinforcement learning.2019

    • Author(s)
      Honda Naoki
    • Organizer
      NSI workshop 2019 ~Neural Mechanisms underlying Behavior~
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Inverse Reinforcement Learning and Active Inference in Gradient Sensing Behavior2019

    • Author(s)
      Honda Naoki
    • Organizer
      Computational Principles in Active Perception and Reinforcement Learning in the Brain
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Identification of Animal Behavioral Strategies by Inverse Reinforcement Learning2018

    • Author(s)
      Naoki Honda
    • Organizer
      神経科学会
    • Related Report
      2018 Research-status Report
  • [Presentation] Identification of Animal Behavioral Strategies by Inverse Reinforcement Learning2018

    • Author(s)
      本田直樹
    • Organizer
      Mortor Contol研究会
    • Related Report
      2018 Research-status Report
    • Invited
  • [Presentation] 動物の行動戦略を解読する逆強化学習~線虫行動への応用~2018

    • Author(s)
      本田直樹、池田宗樹
    • Organizer
      定量生物学の会
    • Related Report
      2018 Research-status Report
    • Invited
  • [Presentation] Chemotactic mechanism for topographic map formation2017

    • Author(s)
      Honda Naoki
    • Organizer
      日本神経科学大会
    • Related Report
      2017 Research-status Report
  • [Presentation] Chemotactic implementation of topographic map formation2017

    • Author(s)
      Honda Naoki
    • Organizer
      日本神経回路学会全国大会
    • Related Report
      2017 Research-status Report
  • [Presentation] トポグラフィックな神経軸索投射の走化性メカニズム2017

    • Author(s)
      本田 直樹
    • Organizer
      日本数理生物学会年会
    • Related Report
      2017 Research-status Report
  • [Presentation] Behavioral analysis of C. elegans by inverse reinforcement learning.2016

    • Author(s)
      Yamaguchi S, Ikeda M, Nakano S, Mori I, Ishii S and Naoki H
    • Organizer
      International Conference on Systems Biology
    • Place of Presentation
      Barcelona, Spain
    • Year and Date
      2016-09-16
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] New measures in FRET imaging2016

    • Author(s)
      Naoki H and Yamao M
    • Organizer
      International Conference on Systems Biology
    • Place of Presentation
      Barcelona, Spain
    • Year and Date
      2016-09-16
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Decoding of intracellular signal transfer from FRET imaging: distinct functions of Rac1 and Cdc42 in cell migration, The 11th AIMS Conference on Dynamical Systems, Differential Equations and Applications2016

    • Author(s)
      Naoki H, Yamao M, Ishii S
    • Organizer
      The 11th AIMS Conference on Dynamical Systems, Differential Equations and Applications
    • Place of Presentation
      Orland, USA
    • Year and Date
      2016-07-01
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research / Invited
  • [Book] AI導入によるバイオテクノロジーの発展2018

    • Author(s)
      本田直樹(共著)
    • Total Pages
      234
    • Publisher
      シーエムシー出版
    • ISBN
      9784781313153
    • Related Report
      2017 Research-status Report
  • [Book] 実験医学2017年5月号2017

    • Author(s)
      篠原美都, 本田直樹, 篠原隆司
    • Total Pages
      141
    • Publisher
      羊土社
    • ISBN
      9784758101639
    • Related Report
      2017 Research-status Report
  • [Remarks] 理論生物学分野

    • URL

      https://sites.google.com/view/theoretical-biology/

    • Related Report
      2018 Research-status Report 2017 Research-status Report

URL: 

Published: 2016-04-21   Modified: 2021-02-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi