Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published May 29, 2007 | Supplemental Material + Published
Journal Article Open

Neural signature of fictive learning signals in a sequential investment task

Abstract

Reinforcement learning models now provide principled guides for a wide range of reward learning experiments in animals and humans. One key learning (error) signal in these models is experiential and reports ongoing temporal differences between expected and experienced reward. However, these same abstract learning models also accommodate the existence of another class of learning signal that takes the form of a fictive error encoding ongoing differences between experienced returns and returns that "could-have-been-experienced" if decisions had been different. These observations suggest the hypothesis that, for all real-world learning tasks, one should expect the presence of both experiential and fictive learning signals. Motivated by this possibility, we used a sequential investment game and fMRI to probe ongoing brain responses to both experiential and fictive learning signals generated throughout the game. Using a large cohort of subjects (n = 54), we report that fictive learning signals strongly predict changes in subjects' investment behavior and correlate with fMRI signals measured in dopaminoceptive structures known to be involved in valuation and choice.

Additional Information

© 2007 by The National Academy of Sciences of the USA Edited by Dale Purves, Duke University Medical Center, Durham, NC, and approved April 13, 2007 (received for review October 6, 2006) This article is a PNAS Direct Submission. Freely available online through the PNAS open access option. We thank B. King-Casas, P. Chiu, and X. Cui for comments on this manuscript; D. Eagleman for stimulating discussions; the Hyperscan Development Team at Baylor College of Medicine for software implementation [NEMO (www.hnl.bcm.tmc.edu/nemo)]; N. Apple, K. Pfeiffer, J. McGee, C. Bracero, X. Cui [xjView (http://people.hnl.bcm.tmc.edu/cuixu/xjView)], and P. Baldwin for technical assistance; and the three anonymous referees for their comments. This work was supported by National Institute on Drug Abuse Grant DA11723 (to P.R.M.), National Institute of Neurological Disorders and Stroke Grant NS045790 (to P.R.M.), and the Kane Family Foundation (P.R.M.). P.R.M. was also supported by the Institute for Advanced Study (Princeton, NJ) for part of the work contained in this article. Author contributions: T.L., K.M., and P.R.M. designed research; T.L. performed research; T.L. and P.R.M. analyzed data; and T.L., K.M., C.F.C., and P.R.M. wrote the paper. Conflict of interest statement: T.L. is Executive Vice President and Director of Research for Computational Management, Inc. This article contains supporting information online at www.pnas.org/cgi/content/full/0608842104/DC1.

Attached Files

Published - LOHpnas07OA.pdf

Supplemental Material - LOHpnas07dataset.pdf

Supplemental Material - LOHpnas07suppfig7.pdf

Supplemental Material - LOHpnas07suppfig8.pdf

Supplemental Material - LOHpnas07suppfig9.pdf

Files

LOHpnas07suppfig7.pdf
Files (1.9 MB)
Name Size Download all
md5:848411cb9973e4fb1ce355ecaa3ced16
690.1 kB Preview Download
md5:07c6981c6c85de506feb1211114d2c07
28.5 kB Preview Download
md5:8d7bd5c6a0670695b8b06cdfc5b0000e
54.8 kB Preview Download
md5:233964004928ed2a41a62125f6db35f6
178.4 kB Preview Download
md5:78d6e9d0257b6dbbc57e2a6f63e74e57
992.8 kB Preview Download

Additional details

Created:
August 22, 2023
Modified:
October 16, 2023