LAPTNet-FPN: Multi-scale LiDAR-aided Projective Transform Network for Real Time Semantic Grid Prediction

TL;DR

This paper introduces LAPTNet-FPN, a multi-scale LiDAR-aided network that improves real-time semantic grid prediction by fusing sensor data, achieving significant accuracy gains and 25 FPS performance.

Contribution

The novel multi-scale LiDAR-aided perspective transform network enhances semantic grid prediction accuracy and efficiency in autonomous systems.

Findings

8.67% improvement in human class accuracy

49.07% improvement in movable object class accuracy

25 FPS inference speed

Abstract

Semantic grids can be useful representations of the scene around an autonomous system. By having information about the layout of the space around itself, a robot can leverage this type of representation for crucial tasks such as navigation or tracking. By fusing information from multiple sensors, robustness can be increased and the computational load for the task can be lowered, achieving real time performance. Our multi-scale LiDAR-Aided Perspective Transform network uses information available in point clouds to guide the projection of image features to a top-view representation, resulting in a relative improvement in the state of the art for semantic grid generation for human (+8.67%) and movable object (+49.07%) classes in the nuScenes dataset, as well as achieving results close to the state of the art for the vehicle, drivable area and walkway classes, while performing inference at 25 FPS.