Derivation of a learning principle from irrational behaviors

• Project/Area Number: 21700174
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Single-year Grants
• Research Field: Intelligent informatics
• Research Institution: Tamagawa University
• Principal Investigator: SAKAI Yutaka 玉川大学, 脳科学研究所, 准教授 (70323376)
• Project Period (FY): 2009 – 2012
• Project Status: Completed (Fiscal Year 2012)
• Budget Amount: ¥3,250,000 (Direct Cost: ¥2,500,000、Indirect Cost: ¥750,000); Fiscal Year 2012: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000); Fiscal Year 2011: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000); Fiscal Year 2010: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000); Fiscal Year 2009: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
• Keywords: 遅延割引 / 学習行動 / 強化学習 / 割引価値問題 / マッチング行動 / 衝動的選好 / 知能機械 / 神経科学 / 動物行動 / 知識獲得

Research Abstract

Animals, including human, apt to prefer immediate returns.The learning framework incorpolating such a behavioral property is established asthe reinforcement learing theory. However, we showed that the applicable range ofthe framework is small and outside of some simple examples of behavioralexperiments of animals. We constructed a novel framework that is always applicablefor animal behaviors and demonstrated that the proposed framework reproduces animal behaviors.

Research Products

• [Journal Article] Reinforcement learning for discounted values often loses the goal in the application to animal learning (2012)
  • Author(s): Yamaguchi Y, Sakai Y
  • Journal Title: Neural networks : the official journal of the International Neural Network Society Volume: 35C Pages: 88-91
  • DOI: 10.1016/j.neunet.2012.08.004
  • Peer Reviewed
• [Journal Article] Reinforcing operandum: rapid andreliable learning of skilled forelimbmovements by head-fixed rodents (2012)
  • Author(s): Kimura R, et al. Sakai Y
  • Journal Title: Journal of Neurophysiology Volume: vol.108 Issue: 6 Pages: 1781-1792
  • DOI: 10.1152/jn.00356.2012
  • Peer Reviewed
• [Journal Article] A theoreticalapproach to animal's impulsivepreference: Impulsive choice behavioris interpreteded as a result of rewardmaximization failure (2012)
  • Author(s): Yamaguchi Y, Sakai Y
  • Journal Title: Proceedings ofSCIS-ISIS2012, Kobe, Japan Volume: Nov. 20-24 Pages: 1182-1185
  • DOI: 10.1109/scis-isis.2012.6505271
  • Peer Reviewed
• [Journal Article] シナプス可塑性と学習-新たな学習原理の発見へ (2010)
  • Author(s): 酒井裕
  • Journal Title: 未来材料 Volume: 10 Pages: 67-72
• [Presentation] Impulsivepreference emerges as a result fromreward-maximization failure (2012)
  • Author(s): Yamaguchi Y, Sakai Y
  • Organizer: 第35回日本神経科学大会
  • Place of Presentation: 名古屋
  • Year and Date: 2012-09-19
• [Presentation] 割引価値問題は被験者の戦略によって不良設定問題となる (2011)
  • Author(s): 山口良哉,酒井裕
  • Organizer: 日本神経回路学会 第21回全国大会
  • Place of Presentation: 沖縄
  • Year and Date: 2011-12-15
• [Presentation] 割引価値問題は被験者の戦略によって不良設定問題となる (2011)
  • Author(s): 酒井裕
  • Organizer: 日本神経回路学会第21回全国大会
  • Place of Presentation: 沖縄科学技術大学院大学(沖縄県)
  • Year and Date: 2011-12-15
• [Presentation] 非合理行動の背後にある合理的学習戦略 (2011)
  • Author(s): 酒井裕
  • Organizer: 日本基礎心理学会 第30回大会
  • Place of Presentation: 横浜
  • Year and Date: 2011-12-04
• [Presentation] 非合理行動の背後にある合理的な学習戦略 (2011)
  • Author(s): 酒井裕
  • Organizer: 日本基礎心理学会第30回大会
  • Place of Presentation: 慶義塾大学日吉キャンパス(神奈川県)(招待講演)
  • Year and Date: 2011-12-04
• [Presentation] The reward-maximization learning indifferent to historical state reproduces the preference reversal in intertemporal choice (2011)
  • Author(s): Sakai Y
  • Organizer: The 34th annual meeting of the Japan neuroscience society
  • Place of Presentation: パシフィコ横浜(神奈川県)
  • Year and Date: 2011-09-17
• [Presentation] Discounted value problem becomes ill-posed by subject's strategy (2011)
  • Author(s): Sakai Y
  • Organizer: The 8th IBRO World Congress of Neuroscience
  • Place of Presentation: Fortezza da Basso (Florence, Italy)
  • Year and Date: 2011-07-17
• [Presentation] Discountedvalue problem becomes ill-posed bysubject's strategy (2011)
  • Author(s): Yamaguchi Y, Sakai Y
  • Organizer: 8th IBRO
  • Place of Presentation: Florence, Italy
  • Year and Date: 2011-07-16
• [Presentation] 強化学習における時間割引の再考 (2011)
  • Author(s): 山口良哉,酒井裕
  • Organizer: 日本物理学会 第66回年次大会
  • Place of Presentation: 新潟
  • Year and Date: 2011-03-25
• [Presentation] 強化学習における時間割引の再考 (2011)
  • Author(s): 酒井裕
  • Organizer: 日本物理学会 第66回年次大会
  • Place of Presentation: 新潟大学(新潟県)
  • Year and Date: 2011-03-25
• [Presentation] Purpose orStrategy? Reconsideration of temporaldiscount in non-Markov situation (2010)
  • Author(s): Yamaguchi Y, Sakai Y
  • Organizer: 第33回日本神経科学大会
  • Place of Presentation: 神戸
  • Year and Date: 2010-09-04
• [Presentation] Purpose or Strategy? Reconsideration of temporal discount in non-Markov situation (2010)
  • Author(s): Sakai, Y.
  • Organizer: Neuro 2010
  • Place of Presentation: 神戸コンベンションセンター(兵庫県)
  • Year and Date: 2010-09-04
• [Presentation] A hypothesis ofefficient learning rule: dopaminedependent metaplasticity (2010)
  • Author(s): Yutaka Sakai
  • Organizer: 第33回日本神経科学大会
  • Place of Presentation: 神戸
  • Year and Date: 2010-09-03
• [Presentation] A hypothesis of efficient learning rule : dopamine-dependent metaplasticity (2010)
  • Author(s): Sakai, Y.
  • Organizer: Neuro 2010
  • Place of Presentation: 神戸コンベンションセンター(兵庫県)
  • Year and Date: 2010-09-03
• [Presentation] A theoretical approach to animal's impulsive preference - Impulsive choice behavior is interpreted as a result of reward-maximization failure
  • Author(s): Yoshiya Yamaguchi
  • Organizer: SCIS-ISIS2012
  • Place of Presentation: 神戸コンベンションセンター（兵庫県）
  • Invited
• [Presentation] Impulsive preference emerges as a result from reward-maximization failure
  • Author(s): Yoshiya Yamaguchi
  • Organizer: Neuroscience 2012
  • Place of Presentation: 名古屋国際会議場（愛知県）
• [Remarks]
  • URL: http://spike.lab.tamagawa.ac.jp/