Visual Knowledge Tracing
Abstract
Each year, thousands of people learn new visual categorization tasks -- radiologists learn to recognize tumors, birdwatchers learn to distinguish similar species, and crowd workers learn how to annotate valuable data for applications like autonomous driving. As humans learn, their brain updates the visual features it extracts and attend to, which ultimately informs their final classification decisions. In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks. We propose models that jointly extract the visual features used by learners as well as predicting the classification functions they utilize. We collect three challenging new datasets from real human learners in order to evaluate the performance of different visual knowledge tracing methods. Our results show that our recurrent models are able to predict the classification behavior of human learners on three challenging medical image and species identification tasks.
Additional Information
Attribution 4.0 International (CC BY 4.0). Thanks to the anonymous reviews for their valuable feedback. This work was in part supported by the Turing 2.0 'Enabling Advanced Autonomy' project funded by the EPSRC and the Alan Turing Institute and also by the Simons Collaboration on the Global Brain.Attached Files
Accepted Version - 2207.10157.pdf
Files
Name | Size | Download all |
---|---|---|
md5:8659a659b9ca28af3fdc897afd240776
|
8.7 MB | Preview Download |
Additional details
- Eprint ID
- 118460
- Resolver ID
- CaltechAUTHORS:20221219-234035307
- Engineering and Physical Sciences Research Council (EPSRC)
- Alan Turing Institute
- Simons Foundation
- Created
-
2022-12-21Created from EPrint's datestamp field
- Updated
-
2023-06-02Created from EPrint's last_modified field