A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Maps

TL;DR

This paper introduces a quantized neural network for depth completion from sparse ToF data, achieving high-quality dense depth maps with significantly reduced memory and computational requirements.

Contribution

The authors propose a low memory footprint, quantized encoder-decoder network for depth completion, optimized for sparse ToF data with minimal quality loss.

Findings

Achieves up to 14-fold reduction in model size.

Maintains state-of-the-art depth quality.

Requires significantly less GPU time.

Abstract

Sparse active illumination enables precise time-of-flight depth sensing as it maximizes signal-to-noise ratio for low power budgets. However, depth completion is required to produce dense depth maps for 3D perception. We address this task with realistic illumination and sensor resolution constraints by simulating ToF datasets for indoor 3D perception with challenging sparsity levels. We propose a quantized convolutional encoder-decoder network for this task. Our model achieves optimal depth map quality by means of input pre-processing and carefully tuned training with a geometry-preserving loss function. We also achieve low memory footprint for weights and activations by means of mixed precision quantization-at-training techniques. The resulting quantized models are comparable to the state of the art in terms of quality, but they require very low GPU times and achieve up to 14-fold memory size reduction for the weights w.r.t. their floating point counterpart with minimal impact on quality metrics.