Deep Imitative Models for Flexible Inference, Planning, and Control

TL;DR

This paper introduces Imitative Models that blend imitation learning and planning to enable flexible, goal-directed autonomous behavior, outperforming existing methods in simulated driving tasks.

Contribution

The paper proposes a novel probabilistic modeling framework that combines imitation learning with goal-directed planning, allowing flexible goal specification and robust autonomous control.

Findings

Outperforms six imitation learning methods and a planning approach in simulated driving.

Learns efficiently from expert demonstrations without online data collection.

Robust to poorly specified goals, such as incorrect side of the road.

Abstract

Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals. Yet, reward functions that evoke desirable behavior are often difficult to specify. In this paper, we propose Imitative Models to combine the benefits of IL and goal-directed planning. Imitative Models are probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals. We derive families of flexible goal objectives, including constrained goal regions, unconstrained goal sets, and energy-based goals. We show that our method can use these objectives to successfully direct behavior. Our method substantially outperforms six IL approaches and a planning-based approach in a dynamic simulated autonomous driving task, and is efficiently learned from expert demonstrations without online data collection. We also show our approach is robust to poorly specified goals, such as goals on the wrong side of the road.