MonoRelief V2: Leveraging Real Data for High-Fidelity Monocular Relief Recovery

TL;DR

MonoRelief V2 is an advanced model that leverages both synthetic and real data, including pseudo-real images generated by AI, to accurately recover 2.5D reliefs from single images with complex variations.

Contribution

It introduces a novel training approach combining pseudo-real and real data, significantly improving relief recovery accuracy over previous synthetic-only models.

Findings

Achieves state-of-the-art depth and normal prediction performance

Effectively utilizes pseudo-real images for training

Demonstrates robustness across complex material and illumination variations

Abstract

This paper presents MonoRelief V2, an end-to-end model designed for directly recovering 2.5D reliefs from single images under complex material and illumination variations. In contrast to its predecessor, MonoRelief V1 [1], which was solely trained on synthetic data, MonoRelief V2 incorporates real data to achieve improved robustness, accuracy and efficiency. To overcome the challenge of acquiring large-scale real-world dataset, we generate approximately 15,000 pseudo real images using a text-to-image generative model, and derive corresponding depth pseudo-labels through fusion of depth and normal predictions. Furthermore, we construct a small-scale real-world dataset (800 samples) via multi-view reconstruction and detail refinement. MonoRelief V2 is then progressively trained on the pseudo-real and real-world datasets. Comprehensive experiments demonstrate its state-of-the-art performance both in depth and normal predictions, highlighting its strong potential for a range of downstream applications. Code is at: https://github.com/glp1001/MonoreliefV2.