On the Sample Complexity of the Linear Quadratic Regulator
Abstract
This paper addresses the optimal control problem known as the linear quadratic regulator in the case when the dynamics are unknown. We propose a multistage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then designs a controller using both the model and uncertainty estimate. Our technique uses contemporary tools from random matrix theory to bound the error in the estimation procedure. We also employ a recently developed approach to control synthesis called System Level Synthesis that enables robust control design by solving a quasi-convex optimization problem. We provide end-to-end bounds on the relative error in control cost that are optimal in the number of parameters and that highlight salient properties of the system to be controlled such as closed-loop sensitivity and optimal control magnitude. We show experimentally that the Coarse-ID approach enables efficient computation of a stabilizing controller in regimes where simple control schemes that do not take the model uncertainty into account fail to stabilize the true system.
Additional Information
© 2019 SFoCM. Received 08 February 2018; Revised 03 December 2018; Accepted 22 February 2019; First Online 05 August 2019. This work was generously supported in part by NSF award CCF-1359814, ONR awards N00014-14-1-0024 and N00014-17-1-2191, the DARPA Fundamental Limits of Learning Program, a Sloan Research Fellowship, and a Google Faculty Award. SD is additionally supported by an NSF Graduate Research Fellowship under Grant No. DGE 1752814. NM is additionally supported by grants from the AFOSR and NSF, and by gifts from Huawei and Google. We thank Ross Boczar, Qingqing Huang, Laurent Lessard, Michael Littman, Manfred Morari, Andrew Packard, Anders Rantzer, Daniel Russo, and Ludwig Schmidt for many helpful comments and suggestions. We also thank the anonymous referees for making several suggestions that have significantly improved the paper and its presentation.Attached Files
Submitted - 1710.01688.pdf
Files
Name | Size | Download all |
---|---|---|
md5:ad6b77c96c0cc6234f61d158f77b146a
|
1.2 MB | Preview Download |
Additional details
- Eprint ID
- 97674
- Resolver ID
- CaltechAUTHORS:20190806-130716007
- CCF-1359814
- NSF
- N00014-14-1-0024
- Office of Naval Research (ONR)
- N00014-17-1-2191
- Office of Naval Research (ONR)
- Defense Advanced Research Projects Agency (DARPA)
- Alfred P. Sloan Foundation
- Google Faculty Research Award
- DGE-1752814
- NSF Graduate Research Fellowship
- Huawei
- Created
-
2019-08-07Created from EPrint's datestamp field
- Updated
-
2021-11-16Created from EPrint's last_modified field