Active Learning under Label Shift
Abstract
We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks.
Additional Information
© 2021 by the author(s). Anqi Liu is supported by the PIMCO Postdoctoral Fellowship. Prof. Anandkumar is supported by Bren endowed Chair, faculty awards from Microsoft, Google, and Adobe, Beyond Limits, and LwLL grants. This work is also supported by funding from Raytheon and NASA TRISH.Attached Files
Published - zhao21b.pdf
Submitted - 2007.08479.pdf
Supplemental Material - zhao21b-supp.pdf
Files
Additional details
- Eprint ID
- 106577
- Resolver ID
- CaltechAUTHORS:20201110-074357009
- PIMCO Postdoctoral Fellowship
- Bren Professor of Computing and Mathematical Sciences
- Microsoft Faculty Fellowship
- Google Faculty Research Award
- Adobe
- Learning with Less Labels (LwLL)
- Raytheon Company
- NASA
- Created
-
2020-11-10Created from EPrint's datestamp field
- Updated
-
2023-06-02Created from EPrint's last_modified field
- Caltech groups
- Center for Autonomous Systems and Technologies (CAST)