LADIS: Language Disentanglement for 3D Shape Editing

TL;DR

This paper introduces LADIS, a novel approach for text-driven 3D shape editing that enhances local, decoupled modifications by learning disentangled language and geometry representations, outperforming previous methods.

Contribution

We propose a new network architecture, a disentanglement loss, and a part-wise edit precision metric to improve local 3D shape editing guided by natural language.

Findings

Outperforms SOTA by 20% in edit locality

Achieves up to 6.6% improvement in language reference resolution

Enables more localized shape edits without explicit part supervision

Abstract

Natural language interaction is a promising direction for democratizing 3D shape design. However, existing methods for text-driven 3D shape editing face challenges in producing decoupled, local edits to 3D shapes. We address this problem by learning disentangled latent representations that ground language in 3D geometry. To this end, we propose a complementary tool set including a novel network architecture, a disentanglement loss, and a new editing procedure. Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision. We show that our method outperforms existing SOTA methods by 20% in terms of edit locality, and up to 6.6% in terms of language reference resolution accuracy. Our work suggests that by solely disentangling language representations, downstream 3D shape editing can become more local to relevant parts, even if the model was never given explicit part-based supervision.