TrACT: A Training Dynamics Aware Contrastive Learning Framework for Long-tail Trajectory Prediction

TL;DR

This paper introduces TrACT, a contrastive learning framework that leverages training dynamics and contextual information to improve long-tail trajectory prediction accuracy and scene compliance in autonomous driving.

Contribution

It proposes a novel two-stage training process incorporating richer contextual features and training dynamics into a contrastive learning framework for better long-tail prediction.

Findings

Achieves state-of-the-art performance on large-scale datasets.

Improves accuracy and scene compliance on challenging long-tail scenarios.

Reduces training bias in trajectory prediction models.

Abstract

As a safety critical task, autonomous driving requires accurate predictions of road users' future trajectories for safe motion planning, particularly under challenging conditions. Yet, many recent deep learning methods suffer from a degraded performance on the challenging scenarios, mainly because these scenarios appear less frequently in the training data. To address such a long-tail issue, existing methods force challenging scenarios closer together in the feature space during training to trigger information sharing among them for more robust learning. These methods, however, primarily rely on the motion patterns to characterize scenarios, omitting more informative contextual information, such as interactions and scene layout. We argue that exploiting such information not only improves prediction accuracy but also scene compliance of the generated trajectories. In this paper, we propose to incorporate richer training dynamics information into a prototypical contrastive learning framework. More specifically, we propose a two-stage process. First, we generate rich contextual features using a baseline encoder-decoder framework. These features are split into clusters based on the model's output errors, using the training dynamics information, and a prototype is computed within each cluster. Second, we retrain the model using the prototypes in a contrastive learning framework. We conduct empirical evaluations of our approach using two large-scale naturalistic datasets and show that our method achieves state-of-the-art performance by improving accuracy and scene compliance on the long-tail samples. Furthermore, we perform experiments on a subset of the clusters to highlight the additional benefit of our approach in reducing training bias.