Pygmalion Effect in Vision: Image-to-Clay Translation for Reflective Geometry Reconstruction

TL;DR

This paper introduces a novel image-to-clay translation framework that improves 3D reflective object reconstruction by suppressing specular reflections and enhancing geometric accuracy through a dual-branch neural network.

Contribution

It proposes a dual-branch network with clay-guided translation to stabilize geometry and refine surface normals in reflective 3D reconstruction tasks.

Findings

Significant improvement in normal accuracy over existing methods

Enhanced mesh completeness in synthetic and real datasets

Effective suppression of specular cues in multi-view images

Abstract

Understanding reflection remains a long-standing challenge in 3D reconstruction due to the entanglement of appearance and geometry under view-dependent reflections. In this work, we present the Pygmalion Effect in Vision, a novel framework that metaphorically "sculpts" reflective objects into clay-like forms through image-to-clay translation. Inspired by the myth of Pygmalion, our method learns to suppress specular cues while preserving intrinsic geometric consistency, enabling robust reconstruction from multi-view images containing complex reflections. Specifically, we introduce a dual-branch network in which a BRDF-based reflective branch is complemented by a clay-guided branch that stabilizes geometry and refines surface normals. The two branches are trained jointly using the synthesized clay-like images, which provide a neutral, reflection-free supervision signal that complements the reflective views. Experiments on both synthetic and real datasets demonstrate substantial improvement in normal accuracy and mesh completeness over existing reflection-handling methods. Beyond technical gains, our framework reveals that seeing by unshining, translating radiance into neutrality, can serve as a powerful inductive bias for reflective object geometry learning.