Publications
CONFERENCE (INTERNATIONAL) Safe Reinforcement Learning Using Model Predictive Control with Probabilistic Control Barrier Function
Xun Shen (Osaka University), Akifumi Wachi, Wataru Hashimoto (Osaka University), Kazumune Hashimoto (Osaka University), Shigemasa Takai (Osaka University)
2024 American Control Conference (ACC 2024)
July 10, 2024
Practical applications of reinforcement learning (RL) often demand that the agents explore safety by satisfying designed constraints. The constraints for safety cannot be satisfied with probability 1 when an unbounded stochastic uncertainty is present. One way is to relax the hard constraints into chance constraints and to consider safe RL with chance constraints. This paper addresses safe RL with chance constraints by Model Predictive Control (MPC) with Probabilistic Control Barrier Function (PCBF). MPC with PCBF is used as a function approximator to deliver the policy that satisfies the demanded chance constraints. RL is used to optimize the parameters in MPC with PCBF to improve the closed performance. We prove that using PCBF as a constraint in MPC ensures the safety imposed by the chance constraints. A scenario-based algorithm is designed for the proposed safe RL implementation. A quadrotor system control problem in turbulent conditions has been used as a numerical example to validate the proposed safe RL method.
Paper :
Safe Reinforcement Learning Using Model Predictive Control with Probabilistic Control Barrier Function
(external link)