Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published August 17, 2022 | Submitted + Supplemental Material
Journal Article Open

Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain

Abstract

Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploitation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunction with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospective utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neural substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.

Additional Information

© 2022 Elsevier Inc. Received 8 November 2021, Revised 18 February 2022, Accepted 27 May 2022, Available online 8 July 2022. This work was funded by grants from the US National Science Foundation (1207573) to J.O.D., the Natural Science and Engineering Research Council (RGPIN-2018-05946) to W.C., and the Social Sciences and Humanities Research Council (CGS-MSFSS) to V.M. Author contributions: J.C. and J.P.O. conceived of the experiments and data analysis. J.C., J.P.O., V.M., and W.C. wrote the manuscript. J.C. carried out the fMRI experiment, and V.M. carried out the replication study. The authors declare no competing interests. Data and code availability: Group level t-maps have been deposited at Neurovault and are publicly available as of the date of publication. DOIs are listed in the key resources table. Behavioral data have been deposited at OSF and are publicly available as of the date of publication. DOIs are listed in the key resources table. Task and analysis code are publicly available as of the date of publication. DOIs are listed in the key resources table. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Attached Files

Submitted - 2021.10.13.464279v1.full.pdf

Supplemental Material - 1-s2.0-S0896627322005025-mmc1.pdf

Files

2021.10.13.464279v1.full.pdf
Files (4.2 MB)
Name Size Download all
md5:c2e248a8c3658a10f827ce238af63f42
2.6 MB Preview Download
md5:7317a4c21cde22932a951a54118609ff
1.6 MB Preview Download

Additional details

Created:
August 22, 2023
Modified:
October 23, 2023