LAPTNet: LiDAR-Aided Perspective Transform Network

TL;DR

LAPTNet is a novel neural network architecture that fuses LiDAR and camera data to generate accurate semantic grids for autonomous vehicle perception, improving over camera-only methods without needing depth prediction.

Contribution

The paper introduces LAPTNet, a new architecture that leverages LiDAR data to enhance semantic grid generation without explicit depth estimation.

Findings

Achieves up to 8.8 point improvement over camera-only methods.

Improves semantic grid accuracy for autonomous driving tasks.

Demonstrates effectiveness on NuScenes dataset.

Abstract

Semantic grids are a useful representation of the environment around a robot. They can be used in autonomous vehicles to concisely represent the scene around the car, capturing vital information for downstream tasks like navigation or collision assessment. Information from different sensors can be used to generate these grids. Some methods rely only on RGB images, whereas others choose to incorporate information from other sensors, such as radar or LiDAR. In this paper, we present an architecture that fuses LiDAR and camera information to generate semantic grids. By using the 3D information from a LiDAR point cloud, the LiDAR-Aided Perspective Transform Network (LAPTNet) is able to associate features in the camera plane to the bird's eye view without having to predict any depth information about the scene. Compared to state-of-theart camera-only methods, LAPTNet achieves an improvement of up to 8.8 points (or 38.13%) over state-of-art competing approaches for the classes proposed in the NuScenes dataset validation split.