Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published October 1, 2013 | Published + Supplemental Material
Journal Article Open

Simultaneous modeling of visual saliency and value computation improves predictions of economic choice

Abstract

Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift–diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions.

Additional Information

© 2013 National Academy of Sciences. Edited by Tony Movshon, New York University, New York, NY, and approved August 2, 2013 (received for review March 10, 2013). We thank Ryan Hyon, Allan Wu, Marilyn Loubier, and the Caltech Bookstore for assistance in creating stimuli. We thank EyeQuant, whose software is used here to compute saliency. The research reported here was supported by the Office of Naval Research (via an award made through Johns Hopkins University), by the G. Harold and Leila Y. Mathers Foundation, by the National Science Foundation, and by the L'Oreal–American Association for the Advancement of Science Fellowship for Women in Science program. Author contributions: R.B.T., M.M., and C.K. designed research; R.B.T. performed research; R.B.T. and M.M. contributed new reagents/analytic tools; R.B.T. analyzed data; and R.B.T., M.M., and C.K. wrote the paper. Conflict of interest statement: C.K. and M.M. are on the Scientific Advisory Board of EyeQuant, whose software is used here to compute saliency. A staff member of EyeQuant performed the saliency computation but neither he, nor anyone else from EyeQuant, had any role in study/model design, data collection, data/model analysis, decision to publish, or preparation of the manuscript. This article is a PNAS Direct Submission.

Attached Files

Published - PNAS-2013-Towal-E3858-67.pdf

Supplemental Material - pnas.201304429SI.pdf

Files

pnas.201304429SI.pdf
Files (1.4 MB)
Name Size Download all
md5:b0adf5fb0d84a028cb7756fa32d4afef
383.8 kB Preview Download
md5:59b169e97dcf68fada0f00a62ad6a089
1.0 MB Preview Download

Additional details

Created:
September 15, 2023
Modified:
October 23, 2023