CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning

TL;DR

CoMI-IRL introduces a transformer-based unsupervised method for multi-intention inverse reinforcement learning, effectively inferring reward functions from diverse demonstrations without prior knowledge of behavioral modes, enabling better interpretation and adaptation.

Contribution

The paper presents CoMI-IRL, a novel unsupervised framework that decouples behavior clustering from reward learning, removing the need for prior knowledge of the number of behaviors.

Findings

Outperforms existing MI-IRL methods without prior behavioral knowledge

Enables visual interpretation of behavior relationships

Adapts to unseen behaviors without full retraining

Abstract

Inverse Reinforcement Learning (IRL) seeks to infer reward functions from expert demonstrations. When demonstrations originate from multiple experts with different intentions, the problem is known as Multi-Intention IRL (MI-IRL). Recent deep generative MI-IRL approaches couple behavior clustering and reward learning, but typically require prior knowledge of the number of true behavioral modes $K^*$. This reliance on expert knowledge limits their adaptability to new behaviors, and only enables analysis related to the learned rewards, and not across the behavior modes used to train them. We propose Contrastive Multi-Intention IRL (CoMI-IRL), a transformer-based unsupervised framework that decouples behavior representation and clustering from downstream reward learning. Our experiments show that CoMI-IRL outperforms existing approaches without a priori knowledge of $K^*$ or labels, while allowing for visual interpretation of behavior relationships and adaptation to unseen behavior without full retraining.