Compositional Shielding and Reinforcement Learning for Multi-Agent Systems

Abstract

In this paper, we present the first MA shielding approach that is Deep reinforcement learning has emerged as a powerful tool for truly compositional, does not require online communication, and obtaining high-performance policies. However, the safety of these provides absolute safety guarantees. Concretely, we assume that arXiv:2410.10460v2 [cs.LO] 14 Jun 2025 policies has been a long-standing issue. One promising paradigm to agents observe a subset of all system variables (i.e., operate in a guarantee safety is a shield, which “shields” a policy from making projection of the global state space). We show how to tractably unsafe actions. However, computing a shield scales exponentially synthesize individual shields in low-dimensional projections. The in the number of state variables. This is a particular concern in challenge we need to overcome is that a straightforward general- multi-agent systems with many agents. In this work, we propose a ization of the classical shield synthesis to the MA setting for truly novel approach for multi-agent shielding. We address scalability by independent shields often fails. The reason is that the projection computing individual shields for each agent. The challenge is that removes the potential to coordinate between the agents, but often typical safety specifications are global properties, but the shields of some form of coordination is required. individual agents only ensure local properties. Our key to overcome To address the need for coordination, we get inspiration from this challenge is to apply assume-guarantee reasoning. Specifically, compositional reasoning, which is a powerful approach, allowing we present a sound proof rule that decomposes a (global, complex) to scale up the analysis of distributed systems. The underlying safety specification into (local, simple) obligations for the shields of principle is to construct a correctness proof of multi-component the individual agents. Moreover, we show that app