Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Data

TL;DR

This paper introduces a novel continuous scene representation method using local Deep Implicit Functions for semantic scene completion on LiDAR data, outperforming existing approaches in geometric accuracy.

Contribution

It proposes a continuous, non-voxel-based scene completion approach that encodes raw point clouds into local latent spaces and assembles a global scene from these patches.

Findings

Outperforms state-of-the-art on Semantic KITTI benchmark

Produces dense 3D scene descriptions from sparse LiDAR data

Avoids trade-offs of voxelization by using implicit functions

Abstract

Semantic scene completion is the task of jointly estimating 3D geometry and semantics of objects and surfaces within a given extent. This is a particularly challenging task on real-world data that is sparse and occluded. We propose a scene segmentation network based on local Deep Implicit Functions as a novel learning-based method for scene completion. Unlike previous work on scene completion, our method produces a continuous scene representation that is not based on voxelization. We encode raw point clouds into a latent space locally and at multiple spatial resolutions. A global scene completion function is subsequently assembled from the localized function patches. We show that this continuous representation is suitable to encode geometric and semantic properties of extensive outdoor scenes without the need for spatial discretization (thus avoiding the trade-off between level of scene detail and the scene extent that can be covered). We train and evaluate our method on semantically annotated LiDAR scans from the Semantic KITTI dataset. Our experiments verify that our method generates a powerful representation that can be decoded into a dense 3D description of a given scene. The performance of our method surpasses the state of the art on the Semantic KITTI Scene Completion Benchmark in terms of geometric completion intersection-over-union (IoU).