Inverse Reinforcement Learning: A Control Lyapunov Approach

TL;DR

This paper introduces a novel inverse reinforcement learning method that learns control Lyapunov functions from demonstrations, ensuring stability and leveraging inverse optimality to infer agent intent in continuous environments.

Contribution

The work reformulates IRL as learning control Lyapunov functions, providing stability guarantees and demonstrating effectiveness in goal-directed movement tasks.

Findings

Successfully learns stable control policies from demonstrations.

Guarantees stability of inferred policies through CLF formulation.

Effective in continuous, goal-directed movement environments.

Abstract

Inferring the intent of an intelligent agent from demonstrations and subsequently predicting its behavior, is a critical task in many collaborative settings. A common approach to solve this problem is the framework of inverse reinforcement learning (IRL), where the observed agent, e.g., a human demonstrator, is assumed to behave according to an intrinsic cost function that reflects its intent and informs its control actions. In this work, we reformulate the IRL inference problem to learning control Lyapunov functions (CLF) from demonstrations by exploiting the inverse optimality property, which states that every CLF is also a meaningful value function. Moreover, the derived CLF formulation directly guarantees stability of inferred control policies. We show the flexibility of our proposed method by learning from goal-directed movement demonstrations in a continuous environment.