Probabilistic Model Predictive Safety Certification for Learning-Based Control
Reinforcement learning (RL) methods have demonstrated their efficiency in simulation. However, many of the applications for which RL offers great potential, such as autonomous driving, are also safety critical and require a certified closed-loop behavior in order to meet the safety specifications in...
Saved in:
Published in | IEEE transactions on automatic control Vol. 67; no. 1; pp. 176 - 188 |
---|---|
Main Authors | , , , |
Format | Journal Article |
Language | English |
Published |
New York
IEEE
01.01.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Reinforcement learning (RL) methods have demonstrated their efficiency in simulation. However, many of the applications for which RL offers great potential, such as autonomous driving, are also safety critical and require a certified closed-loop behavior in order to meet the safety specifications in the presence of physical constraints. This article introduces a concept called probabilistic model predictive safety certification (PMPSC), which can be combined with any RL algorithm and provides provable safety certificates in terms of state and input chance constraints for potentially large-scale systems. The certificate is realized through a stochastic tube that safely connects the current system state with a terminal set of states that is known to be safe. A novel formulation allows a recursively feasible real-time computation of such probabilistic tubes, despite the presence of possibly unbounded disturbances. A design procedure for PMPSC relying on Bayesian inference and recent advances in probabilistic set invariance is presented. Using a numerical car simulation, the method and its design procedure are illustrated by enhancing an RL algorithm with safety certificates. |
---|---|
ISSN: | 0018-9286 1558-2523 |
DOI: | 10.1109/TAC.2021.3049335 |