Positive and negative reward coding by serotonin, acetylcholine, and dopamine network.

2011 Fiscal Year Final Research Report

• Project/Area Number: 20300139
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Neurophysiology and muscle physiology
• Research Institution: Kansai Medical University
• Principal Investigator: NAKAMURA Kae 関西医科大学, 医学部, 教授 (40454607)
• Co-Investigator(Kenkyū-buntansha): KOBAYASHI Yasushi 大阪大学, 生命機能研究科, 准教授 (60311198)
• Project Period (FY): 2008 – 2011
• Keywords: ドパミン / セロトニン / アセチルコリン / 強化学習 / 報酬

Research Abstract

The strength of firing of midbrain dopaminergic neurons has been revealed that it carries reward prediction error signal in the framework of reinforcement learning where an agent chooses actions to maximize rewards. The dopamine neurons receive inputs from the pedunculopontine tegmental nucleus(PPTN), where many cholinergic neurons exist, and the dorsal raphe nucleus(DRN), where many serotonergic neurons exist. Thus we measured single neuronal activity of PPTN and DRN while monkeys performed eye movement tasks to obtain reward of different values and the Pavlovian conditioning task in which appetitive and aversive events were imposed. We found that while different PPTN neurons encoded predicted reward and actual rewards separately, DRN neurons often carried predicted and actual rewards by single neurons. Such different reward signals may be fed and integrated in the dopamine neurons to compute reward prediction error signal, which is used for the control of action and decision making.

Research Products

• [Journal Article] 強化学習の神経機構 (2012)
  • Author(s): 小林康
  • Journal Title: ブレインサイエンスレビュー Pages: 127-150
• [Journal Article] Serotonin and dopamine : Unifying affective, activational, and decision functions (2011)
  • Author(s): Cools, R., Nakamura, K., Daw N. D.
  • Journal Title: Neuropsychopharmacology Volume: 36 Pages: 98-113
  • Peer Reviewed
• [Journal Article] A neural correlate of predicted and actual reward-value information in monkey pedunculopontine tegmental and dorsal raphe nucleus during saccade tasks (2011)
  • Author(s): Okada K, Nakamura K, Kobayashi Y
  • Journal Title: Neural Plasticity Pages: 1-21
  • Peer Reviewed
• [Journal Article] Cerebromalacia with epilepsy and cortical blindness in a laboratory Japanese macaque (2010)
  • Author(s): Yoshizawa, K., Nakao, K., Habiro, M., Hayashi, K., Kuwata, M., Uehara, N., Takashi, Y., Nakamura, K., Tsubura, A.
  • Journal Title: Toxicologic Pathology Volume: 38(7) Pages: 1058-1063
  • Peer Reviewed
• [Journal Article] Coding of task reward value in the dorsal raphe nucleus (2010)
  • Author(s): Bromberg-Martin, E. S., Hikosaka, O., Nakamura, K.
  • Journal Title: The Journal of Neuroscience Volume: 30(18) Pages: 6262-6272
  • Peer Reviewed
• [Journal Article] Characterization of oculomotor and visual activities in the primate pedunculopontine tegmental nucleus during visually guided saccade tasks (2009)
  • Author(s): Ken-ichi Okada, Yasushi Kobayashi
  • Journal Title: European Journal of Neuroscience Volume: 30 Pages: 2211-2233
  • Peer Reviewed
• [Journal Article] Different Pedunculopontine Tegmental Neurons Signal Predicted and Actual Task Rewards (2009)
  • Author(s): Ken-ichi Okada, Keisuke Toyama, Yuka Inoue, Tadashi Isa and Yasushi Kobayashi
  • Journal Title: The Journal of Neuroscience Volume: 29 Pages: 4858-4870
  • Peer Reviewed
• [Journal Article] Reward processing of the basal ganglia : Reward function of Pedunculopontine tegmental nucleus (2009)
  • Author(s): Yasushi Kobayashi, Ken-ichi Okada
  • Journal Title: Brain and Nerve神経研究の進歩 Volume: 61 Pages: 397-404
• [Presentation] Saccade-related modulation of rhythmic firing pattern on pedunculopontine tegmental nucleus neurons in behaving monkeys (2011)
  • Author(s): Ken-ichi Okada, Yasushi Kobayashi
  • Organizer: Society for Neuroscience Meeting
  • Place of Presentation: The Walter E. Washington Convention Center(Washington, DC., USA)
  • Year and Date: 2011-11-15
• [Presentation] Comparison of neuronal signals for reward value in appetitive and aversive contexts in the primate lateral hypothalamus area (2011)
  • Author(s): Atsushi Noritake, Kae Nakamura
  • Organizer: Society for Neuroscience Meeting
  • Place of Presentation: The Walter E. Washington Convention Center(Washington, DC., USA)
  • Year and Date: 2011-11-13
• [Presentation] Neuronal activity in the primate dorsal raphe nucleus encodes positive and negative value (2011)
  • Author(s): Kazuko Hayashi, Kazuko Nakao, Ryuichi Matsuzaki, Ken-ichi Okada, Yasushi Kobayashi, Kae Nakamura
  • Organizer: Society for Neuroscience Meeting
  • Place of Presentation: The Walter E. Washington Convention Center(Washington, DC., USA)
  • Year and Date: 2011-11-13
• [Presentation] APPETITIVE AND AVERSIVE CODING BY THE PRIMATE DORSAL RAPHe NUCLEUS (2011)
  • Author(s): Kae Nakamura
  • Organizer: 11th International Conference on Cognitive Neuroscience(ICON XI)
  • Place of Presentation: Mallorca(Spain)
  • Year and Date: 2011-09-28
• [Presentation] Neuronal modulation in appetitive and aversive contexts in the primate lateral hypothalamus (2011)
  • Author(s): Atsushi Noritake, Kae Nakamura
  • Organizer: 日本神経科学会
  • Place of Presentation: パシフィコ横浜(神奈川)
  • Year and Date: 2011-09-17
• [Presentation] Cognitive control of response inhibition in the primate (2011)
  • Author(s): Kazuko Hayashi, Atsushi Noritake, Kae Nakamura
  • Organizer: 日本神経科学会
  • Place of Presentation: パシフィコ横浜(神奈川)
  • Year and Date: 2011-09-17
• [Presentation] The pedunculopontine tegmental nucleus neurons encode predicted reward signal by tonic regular firing and given reward signal phasically (2011)
  • Author(s): Yasushi Kobayashi and Ken-ichi Okada
  • Organizer: 日本神経科学会
  • Place of Presentation: パシフィコ横浜(神奈川)
  • Year and Date: 2011-09-16
• [Presentation] Neural mechanism of reinforcement learning and motivation (2011)
  • Author(s): 小林康
  • Organizer: 上海交通大学BioX-project
  • Place of Presentation: 上海交通大学(上海)
  • Year and Date: 2011-07-23
• [Presentation] Context dependent firing regularity of pedunculopontine tegmental nucleus of behaving monkeys (2010)
  • Author(s): Ken-ichi Okada, Yasushi Kobayashi
  • Organizer: Society for Neuroscience Meeting
  • Place of Presentation: San Diego Convention Center,(San Diego, USA)
  • Year and Date: 2010-11-16
• [Presentation] Neural processing of appetitive and aversive stimuli in the primate dorsal raphe nucleus (2010)
  • Author(s): Hayashi, K., Nakao, K., Okada, K., Kobayashi, Y., Nakamura, K.
  • Organizer: Society for Neuroscience Meeting
  • Place of Presentation: San Diego Convention Center(San Diego, USA)
  • Year and Date: 2010-11-15
• [Presentation] Positive and negative reward value coding in the primate lateral hypothalamus (2010)
  • Author(s): Noritake, A., Nakamura, K.
  • Organizer: Society for Neuroscience Meeting
  • Place of Presentation: San Diego Convention Center(San Diego, USA)
  • Year and Date: 2010-11-15
• [Presentation] 価値に基づく意思決定の神経機構-新たな展開New perspectives on value-based decision making (2010)
  • Author(s): Nakamura, K.
  • Organizer: 日本神経科学会
  • Place of Presentation: 神戸コンベンションセンター(神戸)
  • Year and Date: 2010-09-04
• [Presentation] サル背側縫線核における報酬および嫌悪刺激の情報表現Neuronal coding of rewarding and aversive stimuli in the primate dorsal raphe mucieus (2010)
  • Author(s): Hayashi, K, . Nakao, K., Okada, K., Kobayashi, Y., Nakamura, K.
  • Organizer: 日本神経科学会
  • Place of Presentation: 神戸コンベンションセンター(神戸)
  • Year and Date: 2010-09-04
• [Presentation] サル視床下部外側野における報酬情報表現は部位により異なるRward-coding by the primate lateral hypothalamus is spatially distinct (2010)
  • Author(s): Noritake, A., Nakamura, K.
  • Organizer: 日本神経科学会
  • Place of Presentation: 神戸コンベンションセンター(神戸)
  • Year and Date: 2010-09-04
• [Presentation] Saccadic eye movements as a neural correlate measure of preparatory set in children with ADHD (2010)
  • Author(s): Yuri Kitamura, Yuka Matsuo, Masako Taniike, Ikuko Mohri, Yasushi Kobayashi, Kanehisa Morimoto
  • Organizer: 日本神経科学会
  • Place of Presentation: 神戸コンベンションセンター(神戸)
  • Year and Date: 2010-09-03
• [Presentation] Rhythmic firing of pedunculopontine tegmental nucleus neurons in behaving monkeys (2010)
  • Author(s): Ken-ichi Okada, Yasushi Kobayashi
  • Organizer: 日本神経科学会
  • Place of Presentation: 神戸コンベンションセンター(神戸)
  • Year and Date: 2010-09-03
• [Presentation] The pedunculopontine tegmental nucleus neurons relay predicted and actual reward and context dependent visuomotor information (2010)
  • Author(s): Yasushi Kobayashi, Ken-ichi Okada
  • Organizer: 日本神経科学会
  • Place of Presentation: 神戸コンベンションセンター(神戸)
  • Year and Date: 2010-09-02
• [Presentation] Reward prediction error computation in the pedunculopontine tegmental nucleus Neurons (2009)
  • Author(s): Kobayashi Y
  • Organizer: International Symposium on Mobiligence
  • Place of Presentation: Awaji Yumebutai International Conference Center(淡路)
  • Year and Date: 20091119-21
• [Presentation] The pedunculopontine tegmental nucleus Neurons relay predicted and actual rewarded information (2009)
  • Author(s): Kobayashi Y, Okada K
  • Organizer: International Symposium on Construction and Reconstruction of the brain
  • Place of Presentation: Awaji Yumebutai International Conference Center(淡路)
  • Year and Date: 20091008-10
• [Presentation] Positive and negative value coding in the primate dorsal raphe nucleus (2009)
  • Author(s): Nakamura, K., Kobayashi, Y., Okada, O., Noritake, A., Matsuzaki, R., Nakao, K.
  • Organizer: Society for Neuroscience
  • Place of Presentation: McCormick Place(Chicago, USA)
  • Year and Date: 2009-10-20
• [Presentation] Analyses of the time course of neuronal activity of the pedunculopontine tegmental nucleus in monkeys for reward conditioned saccade task (2009)
  • Author(s): Okada K and Kobayashi Y
  • Organizer: Society for Neuroscience
  • Place of Presentation: McCormick Place(Chicago, USA)
  • Year and Date: 2009-10-20
• [Presentation] Reward-dependent modulation of neuronal activity in the primate ventral striatum (2009)
  • Author(s): Nakamura, K., Nakao, K., Matsuzaki, R.
  • Organizer: 日本神経科学会
  • Place of Presentation: 名古屋国際会議場(名古屋)
  • Year and Date: 2009-09-16
• [Presentation] Reward coding by the primate dorsal raphe neurons is context dependent (2009)
  • Author(s): Nakamura, K., Kobayashi, Y., Okada, O. Matsuzaki, R., Nakao, K.
  • Organizer: 日本神経科学会
  • Place of Presentation: 名古屋国際会議場(名古屋)
  • Year and Date: 2009-09-16
• [Presentation] Computational mechanism of reward prediction error by the Pedunculopontine tegmental nucleus neurons (2009)
  • Author(s): Kobayashi Y, Okada K
  • Organizer: 日本神経科学会
  • Place of Presentation: 名古屋国際会議場(名古屋)
  • Year and Date: 2009-09-16
• [Presentation] Relation of pedunculopontine tegmental nucleus neurons in monkeys to reward prediction and behavior (2009)
  • Author(s): Okada K and Kobayashi Y
  • Organizer: 日本神経科学会
  • Place of Presentation: 名古屋国際会議場(名古屋)
  • Year and Date: 2009-09-16
• [Presentation] Effects of relative and absolute magnitudes of reward on dorsal raphe neuronal activity in monkeys (2008)
  • Author(s): Nakamura, K., Matsumoto, M., Hikosaka, O.
  • Organizer: Society for Neuroscience
  • Place of Presentation: Walter E. Washington Convention Center(Washington, DC., USA)
  • Year and Date: 2008-11-15
• [Presentation] Neural correlates of task performance of the visually guided saccade tasks in the primate pedunculopontine tegmental nucleus (2008)
  • Author(s): Ken-ichi Okada, Yasushi Kobayashi
  • Organizer: Society for Neuroscience
  • Place of Presentation: Walter E. Washington Convention Center(Washington, DC., USA)
  • Year and Date: 2008-11-15
• [Presentation] Neural correlates of task performance in the primate pedunculopontine tegmental nucleus (2008)
  • Author(s): Ken-ichi Okada, Yasushi Kobayashi
  • Organizer: 日本神経科学会
  • Place of Presentation: 東京国際フォーラム(東京)
  • Year and Date: 2008-07-10
• [Presentation] Pedunculopontine Tegmental Nucleus Neurons Signal Predicted and Actual Reward for Reinforcement Learning (2008)
  • Author(s): Yasushi Kobayashi
  • Organizer: ASSC12
  • Place of Presentation: Gis Convention Center, National Taiwan University(Taipei, Taiwan)
  • Year and Date: 2008-06-19
• [Book] Advances in Reinforcement Learning Chapter Title : Reward prediction error computation in the pedunculopontine tegmental nucleus neurons (2011)
  • Author(s): Yasushi Kobayashi and Ken-ichi Okada
  • Total Pages: 157-180
  • Publisher: INTECH
• [Remarks]
  • URL: http://www.kmu.ac.jp/depts/intro/physiol2.html
• [Remarks]
  • URL: http://www7.bpe.es.osaka-u.ac.jp/～yasushi/gyoseki.htm