On Learning Informative Trajectory Embeddings for Imitation, Classification and Regression

TL;DR

This paper introduces a novel trajectory embedding method that captures multiple skills from state-action sequences without reward labels, improving generalization and performance across imitation, classification, clustering, and regression tasks.

Contribution

The paper presents a new embedding approach for trajectories that encodes multiple abilities and demonstrates superior downstream task performance without relying on reward signals.

Findings

Embeddings effectively represent skills across diverse tasks.

The method outperforms traditional trajectory encoding approaches.

Latent space exhibits properties like behavior control and additive structure.

Abstract

In real-world sequential decision making tasks like autonomous driving, robotics, and healthcare, learning from observed state-action trajectories is critical for tasks like imitation, classification, and clustering. For example, self-driving cars must replicate human driving behaviors, while robots and healthcare systems benefit from modeling decision sequences, whether or not they come from expert data. Existing trajectory encoding methods often focus on specific tasks or rely on reward signals, limiting their ability to generalize across domains and tasks. Inspired by the success of embedding models like CLIP and BERT in static domains, we propose a novel method for embedding state-action trajectories into a latent space that captures the skills and competencies in the dynamic underlying decision-making processes. This method operates without the need for reward labels, enabling better generalization across diverse domains and tasks. Our contributions are threefold: (1) We introduce a trajectory embedding approach that captures multiple abilities from state-action data. (2) The learned embeddings exhibit strong representational power across downstream tasks, including imitation, classification, clustering, and regression. (3) The embeddings demonstrate unique properties, such as controlling agent behaviors in IQ-Learn and an additive structure in the latent space. Experimental results confirm that our method outperforms traditional approaches, offering more flexible and powerful trajectory representations for various applications. Our code is available at https://github.com/Erasmo1015/vte.