PartRAG: Retrieval-Augmented Part-Level 3D Generation and Editing

TL;DR

PartRAG is a novel retrieval-augmented framework for part-level 3D generation and editing that improves diversity, accuracy, and editability by integrating a curated part database with a diffusion transformer.

Contribution

It introduces a Hierarchical Contrastive Retrieval module and a part-level editor, enabling diverse retrieval and localized edits without full regeneration.

Findings

Reduces Chamfer Distance from 0.1726 to 0.1528

Raises F-Score from 0.7472 to 0.844 on Objaverse

Achieves interactive edits in 5-8 seconds

Abstract

Single-image 3D generation with part-level structure remains challenging: learned priors struggle to cover the long tail of part geometries and maintain multi-view consistency, and existing systems provide limited support for precise, localized edits. We present PartRAG, a retrieval-augmented framework that integrates an external part database with a diffusion transformer to couple generation with an editable representation. To overcome the first challenge, we introduce a Hierarchical Contrastive Retrieval module that aligns dense image patches with 3D part latents at both part and object granularity, retrieving from a curated bank of 1,236 part-annotated assets to inject diverse, physically plausible exemplars into denoising. To overcome the second challenge, we add a masked, part-level editor that operates in a shared canonical space, enabling swaps, attribute refinements, and compositional updates without regenerating the whole object while preserving non-target parts and multi-view consistency. PartRAG achieves competitive results on Objaverse, ShapeNet, and ABO-reducing Chamfer Distance from 0.1726 to 0.1528 and raising F-Score from 0.7472 to 0.844 on Objaverse-with inference of 38s and interactive edits in 5-8s. Qualitatively, PartRAG produces sharper part boundaries, better thin-structure fidelity, and robust behavior on articulated objects. Code: https://github.com/AIGeeksGroup/PartRAG. Website: https://aigeeksgroup.github.io/PartRAG.