Confirmation bias optimizes reward learning
Abstract
Confirmation bias - the tendency to overweight information that matches prior beliefs or choices - has been shown to manifest even in simple reinforcement learning. In line with recent work, we find that participants learned significantly more from choice-confirming outcomes in a reward-learning task. What is less clear is whether asymmetric learning rates somehow benefit the learner. Here, we combine data from human participants and artificial agents to examine how confirmation-biased learning might improve performance by counteracting decisional and environmental noise. We evaluate one potential mechanism for such noise reduction: visual attention - a demonstrated driver of both value-based choice and predictive learning. Surprisingly, visual attention showed the opposite pattern to confirmation bias, as participants were most likely to fixate on "missed opportunities", slightly dampening the effects of the confirmation bias we observed. Several million simulated experiments with artificial agents showed this bias to be a reward-maximizing strategy compared to several alternatives, but only if disconfirming feedback is not completely ignored - a condition that visual attention may help to enforce.
Additional Information
The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-ND 4.0 International license. Version 1 - March 1, 2021; Version 2 - March 1, 2021; Version 3 - March 2, 2021; Version 4 - March 11, 2021. The authors have declared no competing interest.Attached Files
Submitted - 2021.02.27.433214v4.full.pdf
Files
Name | Size | Download all |
---|---|---|
md5:890ae4a82e291570122925752582ab6b
|
872.8 kB | Preview Download |
Additional details
- Eprint ID
- 108250
- Resolver ID
- CaltechAUTHORS:20210301-125315461
- Created
-
2021-03-01Created from EPrint's datestamp field
- Updated
-
2021-11-16Created from EPrint's last_modified field