Compactly Restrictable Metric Policy Optimization Problems

We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as metric policy optimization problems (MPOPs). Our goal is to establish theoretical results on the well-posedness of MPOPs that can characterize practicall...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on automatic control Vol. 68; no. 5; pp. 3115 - 3122
Main Authors Dorobantu, Victor D., Azizzadenesheli, Kamyar, Yue, Yisong
Format Journal Article
LanguageEnglish
Published New York IEEE 01.05.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Aerospace electronics
Continuous Markov decision processes (MDPs)
Control systems
Dynamic programming
Extraterrestrial measurements
Markov processes
optimal control
Optimization
physical systems
Reinforcement learning
Robots
sampled-data
selection theorems
value iteration
Online AccessGet full text

Cover

More Information
Summary:We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as metric policy optimization problems (MPOPs). Our goal is to establish theoretical results on the well-posedness of MPOPs that can characterize practically relevant continuous control systems. To do so, we define a special class of MPOPs called compactly restrictable MPOPs (CR-MPOPs), which are flexible enough to capture the complex behavior of robotic systems but specific enough to admit solutions using dynamic programming methods such as value iteration. We show how to arrive at CR-MPOPs using forward-invariance. We further show that our theoretical results on CR-MPOPs can be used to characterize feedback linearizable control affine systems.
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2022.3217269