On the Structural Failure of Chamfer Distance in 3D Shape Optimization

TL;DR

This paper reveals that optimizing Chamfer distance in 3D shape tasks can cause gradient collapse, and demonstrates that non-local coupling is essential to prevent this failure, improving shape morphing results.

Contribution

It identifies the gradient-structural cause of Chamfer distance failure and proposes a global coupling mechanism to prevent collapse in 3D shape optimization.

Findings

Shared-basis deformation suppresses collapse in 2D setting.

Differentiable MPM prior reduces Chamfer gap in 3D shape morphing.

Non-local coupling determines success or failure of Chamfer optimization.

Abstract

Chamfer distance is the standard training loss for point cloud reconstruction, completion, and generation, yet directly optimizing it can produce worse Chamfer values than not optimizing it at all. We show that this paradoxical failure is gradient-structural. The per-point Chamfer gradient creates a many-to-one collapse that is the unique attractor of the forward term and cannot be resolved by any local regularizer, including repulsion, smoothness, and density-aware re-weighting. We derive a necessary condition for collapse suppression: coupling must propagate beyond local neighborhoods. In a controlled 2D setting, shared-basis deformation suppresses collapse by providing global coupling; in 3D shape morphing, a differentiable MPM prior instantiates the same principle, consistently reducing the Chamfer gap across 20 directed pairs with a 2.5$\times$ improvement on the topologically complex dragon. The presence or absence of non-local coupling determines whether Chamfer optimization succeeds or collapses. This provides a practical design criterion for any pipeline that optimizes point-level distance metrics.