Neuroscience 2003 Abstract
| Presentation Number: | 518.16 |
|---|---|
| Abstract Title: | How monkeys learn from their decisions: behavior and models for physiology. |
| Authors: |
Lau, B.*1
; Glimcher, P. W.1
1Ctr. for Neural Sci., NYU, New York, NY |
| Primary Theme and Topics |
Cognition and Behavior - Animal Cognition and Behavior -- Learning & memory: Physiology and imaging |
| Session: |
518. Learning & Memory: Physiology & Imaging I Poster |
| Presentation Time: | Monday, November 10, 2003 4:00 PM-5:00 PM |
| Location: | Morial Convention Center - Hall F-I, Board # BB1 |
| Keywords: | REWARD, DECISION MAKING, CHOICE, BASAL GANGLIA |
Decision theorists typically emphasize two decision variables, the probability and magnitude of reward. Empirical research suggests that the basal ganglia (BG) are important for encoding or learning about these variables. To determine how choice behavior in monkeys varied according to these variables, we trained a monkey in a free-choice task based on Herrnstein's Matching Law. Offset of a central target cued the monkey to choose by making a saccadic eye movement to one of two peripheral targets for which rewards were scheduled at different independent rates. If a reward had been scheduled for a target the monkey chose, fluid reward was given. If a reward had been scheduled for a target not chosen, the reward remained available (Sugrue et al. SfN 2001). In a daily session we varied the probability or magnitude of the rewards associated with each target over 3-8 blocks of 100-250 trials. The log of the ratio of choices was linearly related to the log of the ratio of average rewards with slopes typically less than 1. Thus the behavior of the monkey can be parametrically varied according to the probability and magnitude of reward.
We predicted the monkey's decisions using a reinforcement learning model. The model had two free parameters. One determined how recently observed rewards were weighted to estimate the current reward average (eg., Rescorla & Wagner 1972) and one determined the sensitivity of behavior to differences in average rewards. This model fit the monkey's choice behavior well, capturing both steady-state behavior and fluctuations that occurred at block transitions. It has been proposed that the BG participates in encoding and learning about the probability and magnitude of rewards, suggesting that BG neural responses should be correlated with these variables. Future experiments will address this hypothesis by recording single-unit responses in the caudate nucleus.
We predicted the monkey's decisions using a reinforcement learning model. The model had two free parameters. One determined how recently observed rewards were weighted to estimate the current reward average (eg., Rescorla & Wagner 1972) and one determined the sensitivity of behavior to differences in average rewards. This model fit the monkey's choice behavior well, capturing both steady-state behavior and fluctuations that occurred at block transitions. It has been proposed that the BG participates in encoding and learning about the probability and magnitude of rewards, suggesting that BG neural responses should be correlated with these variables. Future experiments will address this hypothesis by recording single-unit responses in the caudate nucleus.
Supported by EY-010536, DOD NDSEG Fellowship
Sample Citation:
[Authors]. [Abstract Title]. Program No. XXX.XX. 2003 Neuroscience Meeting Planner. New Orleans, LA: Society for Neuroscience, 2003. Online.
Copyright © 2003-2026 Society for Neuroscience; all rights reserved. Permission to republish any abstract or part of any abstract in any form must be obtained in writing by SfN office prior to publication.
