IDIL: Imitation Learning of Intent-Driven Expert Behavior

TL;DR

IDIL is a new imitation learning algorithm that effectively models diverse, intent-driven expert behaviors in sequential tasks with high-dimensional states, outperforming existing benchmarks without adversarial training.

Contribution

IDIL introduces an iterative, intent-aware imitation learning method capable of handling complex, high-dimensional sequential tasks and diverse behaviors without adversarial training.

Findings

IDIL matches or exceeds benchmark performance in task metrics.

IDIL provides superior intent inference capabilities.

IDIL captures a broad spectrum of expert behaviors.

Abstract

When faced with accomplishing a task, human experts exhibit intentional behavior. Their unique intents shape their plans and decisions, resulting in experts demonstrating diverse behaviors to accomplish the same task. Due to the uncertainties encountered in the real world and their bounded rationality, experts sometimes adjust their intents, which in turn influences their behaviors during task execution. This paper introduces IDIL, a novel imitation learning algorithm to mimic these diverse intent-driven behaviors of experts. Iteratively, our approach estimates expert intent from heterogeneous demonstrations and then uses it to learn an intent-aware model of their behavior. Unlike contemporary approaches, IDIL is capable of addressing sequential tasks with high-dimensional state representations, while sidestepping the complexities and drawbacks associated with adversarial training (a mainstay of related techniques). Our empirical results suggest that the models generated by IDIL either match or surpass those produced by recent imitation learning benchmarks in metrics of task performance. Moreover, as it creates a generative model, IDIL demonstrates superior performance in intent inference metrics, crucial for human-agent interactions, and aptly captures a broad spectrum of expert behaviors.