Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published May 27, 2010 | Supplemental Material + Accepted Version
Journal Article Open

States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning

Abstract

Reinforcement learning (RL) uses sequential experience with situations ("states") and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model. A state prediction error (SPE) plays a central role, reporting discrepancies between the current model and the observed state transitions. Using functional magnetic resonance imaging in humans solving a probabilistic Markov decision task, we found the neural signature of an SPE in the intraparietal sulcus and lateral prefrontal cortex, in addition to the previously well-characterized RPE in the ventral striatum. This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.

Additional Information

© 2010 Elsevier Inc. Accepted 26 March 2010. Published: May 26, 2010. Available online 26 May 2010. This work was supported in part by the Akademie der Naturforscher Leopoldina LPD Grant 9901/8-140 (J.G.), by grants from the National Institute of Mental Health to J.P.O.D., by grants from the Gordon and Betty Moore Foundation to J.P.O.D. and the Caltech Brain Imaging Center, and by the Gatsby Charitable Foundation (P.D.). The authors declare no financial conflict of interest.

Attached Files

Accepted Version - nihms-199499.pdf

Supplemental Material - mmc1.pdf

Files

mmc1.pdf
Files (2.7 MB)
Name Size Download all
md5:2453092f56cd3c619f71d1e444015994
616.5 kB Preview Download
md5:eb050911ce4ae101cd195b28750d2c78
2.0 MB Preview Download

Additional details

Created:
August 19, 2023
Modified:
October 20, 2023