Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain
Abstract
Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploitation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunction with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospective utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neural substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.
Additional Information
© 2022 Elsevier Inc. Received 8 November 2021, Revised 18 February 2022, Accepted 27 May 2022, Available online 8 July 2022. This work was funded by grants from the US National Science Foundation (1207573) to J.O.D., the Natural Science and Engineering Research Council (RGPIN-2018-05946) to W.C., and the Social Sciences and Humanities Research Council (CGS-MSFSS) to V.M. Author contributions: J.C. and J.P.O. conceived of the experiments and data analysis. J.C., J.P.O., V.M., and W.C. wrote the manuscript. J.C. carried out the fMRI experiment, and V.M. carried out the replication study. The authors declare no competing interests. Data and code availability: Group level t-maps have been deposited at Neurovault and are publicly available as of the date of publication. DOIs are listed in the key resources table. Behavioral data have been deposited at OSF and are publicly available as of the date of publication. DOIs are listed in the key resources table. Task and analysis code are publicly available as of the date of publication. DOIs are listed in the key resources table. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.Attached Files
Submitted - 2021.10.13.464279v1.full.pdf
Supplemental Material - 1-s2.0-S0896627322005025-mmc1.pdf
Files
Name | Size | Download all |
---|---|---|
md5:c2e248a8c3658a10f827ce238af63f42
|
2.6 MB | Preview Download |
md5:7317a4c21cde22932a951a54118609ff
|
1.6 MB | Preview Download |
Additional details
- Alternative title
- Novelty and uncertainty interact to regulate the balance between exploration and exploitation in the human brain
- Eprint ID
- 111483
- Resolver ID
- CaltechAUTHORS:20211015-222205042
- SES-1207573
- NSF
- RGPIN-2018-05946
- Natural Sciences and Engineering Research Council of Canada (NSERC)
- CGS-MSFSS
- Social Sciences and Humanities Research Council (SSHRC)
- Created
-
2021-10-18Created from EPrint's datestamp field
- Updated
-
2022-08-24Created from EPrint's last_modified field
- Caltech groups
- Tianqiao and Chrissy Chen Institute for Neuroscience