Publications
CONFERENCE (INTERNATIONAL) A Survey of Constraint Formulations in Safe Reinforcement Learning
Akifumi Wachi, Xun Shen (Osaka University), Yanan Sui (Tsinghua University)
The 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024)
August 03, 2024
Safety is critical when applying reinforcement learning (RL) to real-world problems. As a result, safe RL has emerged as a fundamental and powerful paradigm for optimizing an agent’s policy while incorporating notions of safety. A prevalent safe RL approach is based on a constrained criterion, which seeks to maximize the expected cumulative reward subject to specific safety constraints. Despite recent effort to enhance safety in RL, a systematic understanding of the field remains difficult. This challenge stems from the diversity of constraint representations and little exploration of their interrelations. To bridge this knowledge gap, we present a comprehensive review of representative constraint formulations, along with a curated selection of algorithms designed specifically for each formulation. In addition, we elucidate the theoretical underpinnings that reveal the mathematical mutual relations among common problem formulations. We conclude with a discussion of the current state and future directions of safe reinforcement learning research.
Paper :
A Survey of Constraint Formulations in Safe Reinforcement Learning
(external link)