PEGAsus: 3D Personalization of Geometry and Appearance

TL;DR

PEGAsus is a novel framework that enables personalized 3D shape generation by learning and composing geometric and appearance attributes from reference shapes with text prompts, allowing fine-grained control and diversity.

Contribution

It introduces a progressive, decoupled learning strategy for shape concepts and extends to region-wise learning, improving personalization and flexibility in 3D shape synthesis.

Findings

Effective attribute extraction from diverse shapes

Flexible composition with text prompts for diverse shapes

Outperforms state-of-the-art methods in experiments

Abstract

We present PEGAsus, a new framework capable of generating Personalized 3D shapes by learning shape concepts at both Geometry and Appearance levels. First, we formulate 3D shape personalization as extracting reusable, category-agnostic geometric and appearance attributes from reference shapes, and composing these attributes with text to generate novel shapes. Second, we design a progressive optimization strategy to learn shape concepts at both the geometry and appearance levels, decoupling the shape concept learning process. Third, we extend our approach to region-wise concept learning, enabling flexible concept extraction, with context-aware and context-free losses. Extensive experimental results show that PEGAsus is able to effectively extract attributes from a wide range of reference shapes and then flexibly compose these concepts with text to synthesize new shapes. This enables fine-grained control over shape generation and supports the creation of diverse, personalized results, even in challenging cross-category scenarios. Both quantitative and qualitative experiments demonstrate that our approach outperforms existing state-of-the-art solutions.