Published May 2007
| public
Book Section - Chapter
Adding Prediction Risk to the Theory of Reward Learning
- Creators
- Preuschoff, Kerstin
- Bossaerts, Peter
Abstract
This article analyzesthe simple Rescorla–Wagner learning rule from the vantage point of least squares learning theory. In particular, it suggests how measures of risk, such as prediction risk, can be used to adjust the learning constant in reinforcement learning. It argues that prediction risk is most effectively incorporated by scaling the prediction errors. This way, the learning rate needs adjusting only when the covariance between optimal predictions and past (scaled) prediction errors changes. Evidence is discussed that suggests that the dopaminergic system in the (human and nonhuman) primate brain encodes prediction risk, and that prediction errors are indeed scaled with prediction risk (adaptive encoding).
Additional Information
© 2007 New York Academy of Sciences. Article first published online: 20 Jun. 2007. We thank John O'Doherty for detailed comments on an earlier draft, and Tim Behrens, Mark Walton, and Matthew Rushworth for further discussions on the link between uncertainty and the learning rate. Peter Bossaerts thanks the Swiss Finance Institute for financial support during his stay at the Université de Lausanne, where this article was written.Additional details
- Eprint ID
- 20430
- Resolver ID
- CaltechAUTHORS:20101014-101918850
- Swiss Finance Institute
- Created
-
2010-10-14Created from EPrint's datestamp field
- Updated
-
2021-11-08Created from EPrint's last_modified field
- Series Name
- Annals of the New York Academy of Sciences
- Series Volume or Issue Number
- 1104