Convergence Rates for Learning Linear Operators from Noisy Data
Abstract
This paper studies the learning of linear operators between infinite-dimensional Hilbert spaces. The training data comprises pairs of random input vectors in a Hilbert space and their noisy images under an unknown self-adjoint linear operator. Assuming that the operator is diagonalizable in a known basis, this work solves the equivalent inverse problem of estimating the operator's eigenvalues given the data. Adopting a Bayesian approach, the theoretical analysis establishes posterior contraction rates in the infinite data limit with Gaussian priors that are not directly linked to the forward map of the inverse problem. The main results also include learning-theoretic generalization error guarantees for a wide range of distribution shifts. These convergence rates quantify the effects of data smoothness and true eigenvalue decay or growth, for compact or unbounded operators, respectively, on sample complexity. Numerical evidence supports the theory in diagonal and nondiagonal settings.
Additional Information
© 2023 Society for Industrial and Applied Mathematics. Funding: The first author is supported by the Simons Foundation under the MATH + X program, U.S. Department of Energy, Office of Basic Energy Sciences, Chemical Sciences, Geosciences, and Biosciences Division under grant DE-SC0020345, the National Science Foundation (NSF) under grant DMS-1815143, and the corporate members of the Geo-Mathematical Imaging Group at Rice University. The third author is supported by the NSF Graduate Research Fellowship Program under grant DGE-1745301. The fourth author is supported by NSF (grant DMS-1818977) and AFOSR (MURI award FA9550-20-1-0358---Machine Learning and Physics-Based Modeling and Simulation). The second, third, and fourth authors are supported by NSF (grant AGS-1835860) and ONR (grant N00014-19-1-2408). The authors thank Kamyar Azizzadenesheli and Joel A. Tropp for helpful discussions about statistical learning. The authors are also grateful to the associate editor and two anonymous referees for their helpful feedback. The computations presented in this paper were conducted on the Resnick High Performance Computing Center, a facility supported by the Resnick Sustainability Institute at the California Institute of Technology.Attached Files
Published - 21m1442942.pdf
Files
Name | Size | Download all |
---|---|---|
md5:1c28b713db2a592713ff9c6c33b28b17
|
1.6 MB | Preview Download |
Additional details
- Eprint ID
- 121794
- Resolver ID
- CaltechAUTHORS:20230613-730765600.19
- Simons Foundation
- Department of Energy (DOE)
- DE-SC0020345
- NSF
- DMS-1815143
- Rice University
- NSF Graduate Research Fellowship
- DGE-1745301
- NSF
- DMS-1818977
- Air Force Office of Scientific Research (AFOSR)
- FA9550-20-1-0358
- NSF
- AGS-1835860
- Office of Naval Research (ONR)
- N00014-19-1-2408
- Resnick Sustainability Institute
- Created
-
2023-07-10Created from EPrint's datestamp field
- Updated
-
2023-07-10Created from EPrint's last_modified field
- Caltech groups
- Resnick Sustainability Institute