Sequential Joint Shape and Pose Estimation of Vehicles with Application to Automatic Amodal Segmentation Labeling

TL;DR

This paper introduces a recurrent neural network-based method that leverages consecutive sensor signals to improve shape and pose estimation of vehicles, enabling automatic high-quality amodal segmentation labeling for autonomous driving.

Contribution

It presents a novel algorithm that uses temporal sensor data for better shape and pose estimation and introduces an automatic annotation pipeline for amodal segmentation labels.

Findings

Improved shape and pose estimation accuracy.

Automatic high-quality amodal segmentation labels generated.

Enhanced understanding of occluded and distant objects.

Abstract

Shape and pose estimation is a critical perception problem for a self-driving car to fully understand its surrounding environment. One fundamental challenge in solving this problem is the incomplete sensor signal (e.g., LiDAR scans), especially for faraway or occluded objects. In this paper, we propose a novel algorithm to address this challenge, which explicitly leverages the sensor signal captured over consecutive time: the consecutive signals can provide more information about an object, including different viewpoints and its motion. By encoding the consecutive signals via a recurrent neural network, not only our algorithm improves the shape and pose estimates, but also produces a labeling tool that can benefit other tasks in autonomous driving research. Specifically, building upon our algorithm, we propose a novel pipeline to automatically annotate high-quality labels for amodal segmentation on images, which are hard and laborious to annotate manually. Our code and data will be made publicly available.