End-to-end Autonomous Vehicle Following System using Monocular Fisheye Camera

TL;DR

This paper presents an end-to-end autonomous vehicle following system using only a monocular fisheye camera, improving performance and general applicability for vehicle platooning without relying on expensive sensors.

Contribution

The paper introduces a novel end-to-end framework with semantic masking and dynamic sampling, enabling robust vehicle following in diverse scenarios using only a monocular fisheye camera.

Findings

Outperforms traditional multi-stage algorithms in real-world tests

Successfully follows vehicles in various scenarios

Demonstrates cost-effective autonomous platooning

Abstract

The increase in vehicle ownership has led to increased traffic congestion, more accidents, and higher carbon emissions. Vehicle platooning is a promising solution to address these issues by improving road capacity and reducing fuel consumption. However, existing platooning systems face challenges such as reliance on lane markings and expensive high-precision sensors, which limits their general applicability. To address these issues, we propose a vehicle following framework that expands its capability from restricted scenarios to general scenario applications using only a camera. This is achieved through our newly proposed end-to-end method, which improves overall driving performance. The method incorporates a semantic mask to address causal confusion in multi-frame data fusion. Additionally, we introduce a dynamic sampling mechanism to precisely track the trajectories of preceding vehicles. Extensive closed-loop validation in real-world vehicle experiments demonstrates the system's ability to follow vehicles in various scenarios, outperforming traditional multi-stage algorithms. This makes it a promising solution for cost-effective autonomous vehicle platooning. A complete real-world vehicle experiment is available at https://youtu.be/zL1bcVb9kqQ.