Redundant Information Neural Estimation
Abstract
We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the "redundant information". We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, we show that optimizing over functions enables the approximation of the redundant information for high-dimensional and continuous predictors. We demonstrate this on high-dimensional image classification and motor-neuroscience tasks.
Additional Information
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Received: 11 May 2021 / Revised: 1 July 2021 / Accepted: 14 July 2021 / Published: 20 July 2021. We thank the reviewers for the helpful comments and suggestions. We thank Parthe Pandit and Hengjie Yang for helpful discussions. We thank Krishna Shenoy and Sergey Stavisky for permission to use the neural recording data sets from a delayed reach task. M.K. was supported by the National Sciences and Engineering Research Council (NSERC). J.C.K. was supported by an NSF CAREER Award (#1943467). This research was supported by a UCLA Computational Medicine Amazon Web Services Award. Author Contributions: Conceptualization, M.K.; methodology, M.K. and A.A.; software, M.K.; writing—original draft preparation, M.K.; writing—review and editing, M.K., A.A., S.S., J.C.K.; supervision, A.A., S.S., J.C.K. All authors have read and agreed to the published version of the manuscript. Data Availability Statement: Code is available at: www.github.com/mjkleinman/RINE. The authors declare no conflict of interest.Attached Files
Published - entropy-23-00922.pdf
Files
| Name | Size | Download all |
|---|---|---|
|
md5:825b2b86cc1d7e84aa07efb89c659bd6
|
1.1 MB | Preview Download |
Additional details
- PMCID
- PMC8304362
- Eprint ID
- 110134
- Resolver ID
- CaltechAUTHORS:20210803-170853656
- Natural Sciences and Engineering Research Council of Canada (NSERC)
- IIS-1943467
- NSF
- Amazon Web Services
- Created
-
2021-08-04Created from EPrint's datestamp field
- Updated
-
2021-08-04Created from EPrint's last_modified field