End-to-End Driving via Self-Supervised Imitation Learning Using Camera and LiDAR Data

TL;DR

This paper introduces a fully self-supervised imitation learning framework for end-to-end autonomous driving that learns from sensor data without requiring human driving commands or pre-trained models, achieving comparable accuracy to supervised methods.

Contribution

The paper presents the first self-supervised learning framework for end-to-end driving that does not rely on human command data or pre-trained models, utilizing pseudo labels derived from vehicle poses.

Findings

Achieves comparable accuracy to supervised learning methods.

Outperforms existing pseudo-label predictors using PID control.

Demonstrates effectiveness across three benchmark datasets.

Abstract

In autonomous driving, the end-to-end (E2E) driving approach that predicts vehicle control signals directly from sensor data is rapidly gaining attention. To learn a safe E2E driving system, one needs an extensive amount of driving data and human intervention. Vehicle control data is constructed by many hours of human driving, and it is challenging to construct large vehicle control datasets. Often, publicly available driving datasets are collected with limited driving scenes, and collecting vehicle control data is only available by vehicle manufacturers. To address these challenges, this letter proposes the first fully self-supervised learning framework, self-supervised imitation learning (SSIL), for E2E driving, based on the self-supervised regression learning (SSRL) framework.The proposed SSIL framework can learn E2E driving networks \emph{without} using driving command data or a pre-trained model. To construct pseudo steering angle data, proposed SSIL predicts a pseudo target from the vehicle's poses at the current and previous time points that are estimated with light detection and ranging sensors. In addition, we propose two E2E driving networks that predict driving commands depending on high-level instruction. Our numerical experiments with three different benchmark datasets demonstrate that the proposed SSIL framework achieves \emph{very} comparable E2E driving accuracy with the supervised learning counterpart. The proposed pseudo-label predictor outperformed an existing one using proportional integral derivative controller.