Changepoint detection versus reinforcement learning: Separable neural substrates approximate different forms of Bayesian inference
- Creators
- Pauli, Wolfgang M.
- Jones, Matt
Abstract
Adaptive behavior in even the simplest decision-making tasks requires predicting future events in an environment that is generally nonstationary. As an inductive problem, this prediction requires a commitment to the statistical process underlying environmental change. This challenge can be formalized in a Bayesian framework as a question of choosing a generative model for the task dynamics. Previous learning models assume, implicitly or explicitly, that nonstationarity follows either a continuous diffusion process or a discrete changepoint process. Each approach is slow to adapt when its assumptions are violated. A new mixture of Bayesian experts framework proposes separable brain systems approximating inference under different assumptions regarding the statistical structure of the environment. This model explains data from a laboratory foraging task, in which rats experienced a change in reward contingencies after pharmacological disruption of dorsolateral (DLS) or dorsomedial striatum (DMS). The data and model suggest DLS learns under a diffusion prior whereas DMS learns under a changepoint prior. The combination of these two systems offers a new explanation for how the brain handles inference in an uncertain environment.
Additional Information
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license. This research was supported by Air Force Office of Scientific Research grant FA9550-10-1-0177. Thanks for helpful discussions of this work go to the members of the computational cognitive neuroscience lab at the university of Colorado at Boulder, and members of the decision neuroscience lab at the California Institute of Technology.Attached Files
Submitted - 591818.full.pdf
Files
Name | Size | Download all |
---|---|---|
md5:080684d7d1d82f8399c5c6a52af1af37
|
865.5 kB | Preview Download |
Additional details
- Eprint ID
- 94259
- Resolver ID
- CaltechAUTHORS:20190328-145236200
- FA9550-10-1-0177
- Air Force Office of Scientific Research (AFOSR)
- Created
-
2019-03-28Created from EPrint's datestamp field
- Updated
-
2021-11-16Created from EPrint's last_modified field