# 3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization

## TL;DR

This paper introduces a novel fusion method combining LiDAR and stereo camera data using a stereo matching network with Conditional Cost Volume Normalization, enhancing depth perception with minimal computational overhead.

## Contribution

The work proposes a generic fusion framework with CCVNorm integrated into stereo matching networks, improving robustness and performance in depth estimation tasks.

## Key findings

- Achieved superior results on KITTI datasets compared to other fusion methods.
- Demonstrated robustness and efficiency with minimal added computational cost.
- Validated the effectiveness of hierarchical CCVNorm extension.

## Abstract

The complementary characteristics of active and passive depth sensing techniques motivate the fusion of the Li-DAR sensor and stereo camera for improved depth perception. Instead of directly fusing estimated depths across LiDAR and stereo modalities, we take advantages of the stereo matching network with two enhanced techniques: Input Fusion and Conditional Cost Volume Normalization (CCVNorm) on the LiDAR information. The proposed framework is generic and closely integrated with the cost volume component that is commonly utilized in stereo matching neural networks. We experimentally verify the efficacy and robustness of our method on the KITTI Stereo and Depth Completion datasets, obtaining favorable performance against various fusion strategies. Moreover, we demonstrate that, with a hierarchical extension of CCVNorm, the proposed method brings only slight overhead to the stereo matching network in terms of computation time and model size. For project page, see https://zswang666.github.io/Stereo-LiDAR-CCVNorm-Project-Page/