3D Road Defect Mapping via Differentiable Neural Rendering and Multi-Frame Semantic Fusion in Bird’s-Eye-View Space

TL;DR

This paper introduces a new method for creating accurate 3D maps of road defects using video and advanced computer vision techniques.

Contribution

The novel framework combines differentiable neural rendering and multi-frame fusion to enable precise 3D road defect mapping.

Findings

The framework reduces detection errors by 33.7% using semantic filtering.

Multi-frame fusion improves accuracy in handling occlusions and motion blur.

The method outperforms single-frame 2D approaches in 3D defect mapping.

Abstract

Road defect detection is essential for traffic safety and infrastructure maintenance. Excising automated methods based on 2D image analysis lack spatial context and cannot provide accurate 3D localization required for maintenance planning. We propose a novel framework for road defect mapping from monocular video sequences by integrating differentiable Bird’s-Eye-View (BEV) mesh representation, semantic filtering, and multi-frame temporal fusion. Our differentiable mesh-based BEV representation enables efficient scene reconstruction from sparse observations through MLP-based optimization. The semantic filtering strategy leverages road surface segmentation to eliminate off-road false positives, reducing detection errors by 33.7%. Multi-frame fusion with ray-casting projection and exponential moving average update accumulates defect observations across frames while maintaining 3D geometric consistency. Experimental results demonstrate that our framework produces geometrically consistent BEV defect maps with superior accuracy compared to single-frame 2D methods, effectively handling occlusions, motion blur, and varying illumination conditions.

Figures