Controlling Steering with Energy-Based Models

TL;DR

This paper evaluates energy-based models for steering control in self-driving cars, comparing them with explicit models, and finds they handle multimodalities slightly better but do not significantly improve driving performance.

Contribution

It provides an extensive comparison of energy-based and explicit behavioral cloning methods for steering control, highlighting their strengths and limitations in real-world driving tasks.

Findings

Energy-based models perform comparably to explicit models in safety interventions.

They exhibit higher jerk, indicating less smooth steering.

Handling multimodalities slightly better does not significantly improve driving ability.

Abstract

So-called implicit behavioral cloning with energy-based models has shown promising results in robotic manipulation tasks. We tested if the method's advantages carry on to controlling the steering of a real self-driving car with an end-to-end driving model. We performed an extensive comparison of the implicit behavioral cloning approach with explicit baseline approaches, all sharing the same neural network backbone architecture. Baseline explicit models were trained with regression (MAE) loss, classification loss (softmax and cross-entropy on a discretization), or as mixture density networks (MDN). While models using the energy-based formulation performed comparably to baseline approaches in terms of safety driver interventions, they had a higher whiteness measure, indicating higher jerk. To alleviate this, we show two methods that can be used to improve the smoothness of steering. We confirmed that energy-based models handle multimodalities slightly better than simple regression, but this did not translate to significantly better driving ability. We argue that the steering-only road-following task has too few multimodalities to benefit from energy-based models. This shows that applying implicit behavioral cloning to real-world tasks can be challenging, and further investigation is needed to bring out the theoretical advantages of energy-based models.