Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published April 2012 | public
Journal Article

Networked Markov Decision Processes With Delays

Abstract

We consider a networked control system, where each subsystem evolves as a Markov decision process with some extra inputs from other systems. Each subsystem is coupled to its neighbors via communication links over which the signals are delayed, but are otherwise transmitted noise-free. A centralized controller receives delayed state information from each subsystem. The control action applied to each subsystem takes effect after a certain delay rather than immediately. We give an explicit bound on the finite history of measurement and control that is required for the optimal control of such networked Markov decision processes. We also show that these bounds depend only on the underlying graph structure as well as the associated delays. Thus, the partially observed Markov decision process associated with a networked Markov decision process can be converted into an information state Markov decision process, whose state does not grow with time.

Additional Information

© 2012 IEEE. Manuscript received January 30, 2009; revised August 18, 2009, May 25, 2010, and June 15, 2010; accepted August 26, 2011. Date of publication September 15, 2011; date of current version March 28, 2012. Recommended by Associate Editor C. Szepesvari.

Additional details

Created:
August 22, 2023
Modified:
October 17, 2023