Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published May 27, 2020 | Submitted
Report Open

Stochastic Finite State Control of POMDPs with LTL Specifications

Abstract

Partially observable Markov decision processes (POMDPs) provide a modeling framework for autonomous decision making under uncertainty and imperfect sensing, e.g. robot manipulation and self-driving cars. However, optimal control of POMDPs is notoriously intractable. This paper considers the quantitative problem of synthesizing sub-optimal stochastic finite state controllers (sFSCs) for POMDPs such that the probability of satisfying a set of high-level specifications in terms of linear temporal logic (LTL) formulae is maximized. We begin by casting the latter problem into an optimization and use relaxations based on the Poisson equation and McCormick envelopes. Then, we propose an stochastic bounded policy iteration algorithm, leading to a controlled growth in sFSC size and an any time algorithm, where the performance of the controller improves with successive iterations, but can be stopped by the user based on time or memory considerations. We illustrate the proposed method by a robot navigation case study.

Attached Files

Submitted - 2001.07679.pdf

Files

2001.07679.pdf
Files (597.7 kB)
Name Size Download all
md5:6aa822a6cdab322f6bbd3015aed65554
597.7 kB Preview Download

Additional details

Created:
August 19, 2023
Modified:
October 20, 2023