Asimovian Adaptive Agents

TL;DR

This paper introduces a method for developing adaptive agents that maintain predictability and safety through formal verification and efficient incremental reverification, enabling rapid adaptation without compromising constraints.

Contribution

It presents novel approaches combining formal verification with online evolutionary learning, including guaranteed preservation of constraints and efficient incremental reverification algorithms.

Findings

Certain learning operators inherently preserve behavioral constraints

Incremental reverification algorithms significantly reduce verification time

Agents can adapt quickly while ensuring safety constraints are maintained

Abstract

The goal of this research is to develop agents that are adaptive and predictable and timely. At first blush, these three requirements seem contradictory. For example, adaptation risks introducing undesirable side effects, thereby making agents' behavior less predictable. Furthermore, although formal verification can assist in ensuring behavioral predictability, it is known to be time-consuming. Our solution to the challenge of satisfying all three requirements is the following. Agents have finite-state automaton plans, which are adapted online via evolutionary learning (perturbation) operators. To ensure that critical behavioral constraints are always satisfied, agents' plans are first formally verified. They are then reverified after every adaptation. If reverification concludes that constraints are violated, the plans are repaired. The main objective of this paper is to improve the efficiency of reverification after learning, so that agents have a sufficiently rapid response time. We present two solutions: positive results that certain learning operators are a priori guaranteed to preserve useful classes of behavioral assurance constraints (which implies that no reverification is needed for these operators), and efficient incremental reverification algorithms for those learning operators that have negative a priori results.