Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Views

TL;DR

HairGuard is a comprehensive framework that enhances 3D vision tasks by accurately recovering soft boundary details like hair, improving depth estimation and view synthesis through innovative data curation, depth refinement, and inpainting techniques.

Contribution

The paper introduces a novel pipeline combining data curation, depth refinement, and inpainting to specifically address soft boundary challenges in 3D vision tasks.

Findings

Achieves state-of-the-art results in depth estimation and view synthesis.

Significantly improves soft boundary detail recovery.

Enhances visual quality in 3D reconstructions.

Abstract

Soft boundaries, like thin hairs, are commonly observed in natural and computer-generated imagery, but they remain challenging for 3D vision due to the ambiguous mixing of foreground and background cues. This paper introduces Guardians of the Hair (HairGuard), a framework designed to recover fine-grained soft boundary details in 3D vision tasks. Specifically, we first propose a novel data curation pipeline that leverages image matting datasets for training and design a depth fixer network to automatically identify soft boundary regions. With a gated residual module, the depth fixer refines depth precisely around soft boundaries while maintaining global depth quality, allowing plug-and-play integration with state-of-the-art depth models. For view synthesis, we perform depth-based forward warping to retain high-fidelity textures, followed by a generative scene painter that fills disoccluded regions and eliminates redundant background artifacts within soft boundaries. Finally, a color fuser adaptively combines warped and inpainted results to produce novel views with consistent geometry and fine-grained details. Extensive experiments demonstrate that HairGuard achieves state-of-the-art performance across monocular depth estimation, stereo image/video conversion, and novel view synthesis, with significant improvements in soft boundary regions.