TransNormal: Dense Visual Semantics for Diffusion-based Transparent Object Normal Estimation

TL;DR

TransNormal is a novel framework that leverages diffusion priors and dense visual semantics to accurately estimate surface normals of transparent objects from monocular images, addressing challenges in optical properties.

Contribution

It introduces a new diffusion-based normal estimation method with dense semantics integration and a physics-based synthetic dataset for transparent labware.

Findings

Reduces mean error by 24.4% on ClearGrasp

Improves 11.25° accuracy by 22.8%

Achieves 15.2% reduction in mean error on ClearPose

Abstract

Monocular normal estimation for transparent objects is critical for laboratory automation, yet it remains challenging due to complex light refraction and reflection. These optical properties often lead to catastrophic failures in conventional depth and normal sensors, hindering the deployment of embodied AI in scientific environments. We propose TransNormal, a novel framework that adapts pre-trained diffusion priors for single-step normal regression. To handle the lack of texture in transparent surfaces, TransNormal integrates dense visual semantics from DINOv3 via a cross-attention mechanism, providing strong geometric cues. Furthermore, we employ a multi-task learning objective and wavelet-based regularization to ensure the preservation of fine-grained structural details. To support this task, we introduce TransNormal-Synthetic, a physics-based dataset with high-fidelity normal maps for transparent labware. Extensive experiments demonstrate that TransNormal significantly outperforms state-of-the-art methods: on the ClearGrasp benchmark, it reduces mean error by 24.4% and improves 11.25{\deg} accuracy by 22.8%; on ClearPose, it achieves a 15.2% reduction in mean error. The code and dataset will be made publicly available at https://longxiang-ai.github.io/TransNormal.