Pomo3D: 3D-Aware Portrait Accessorizing and More

TL;DR

Pomo3D is a novel 3D portrait editing framework that enables explicit, fine-grained, and out-of-distribution accessory customization, including 3D accessory creation from scribbles and extensive portrait editing options.

Contribution

It introduces Pomo3D, a framework for detailed 3D portrait accessorizing and editing, with new modules for accessory creation from scribbles and bias mitigation.

Findings

Enables out-of-distribution accessory appearances on portraits.

Allows creation of 3D accessories from user scribbles.

Provides extensive global and local portrait editing options.

Abstract

We propose Pomo3D, a 3D portrait manipulation framework that allows free accessorizing by decomposing and recomposing portraits and accessories. It enables the avatars to attain out-of-distribution (OOD) appearances of simultaneously wearing multiple accessories. Existing methods still struggle to offer such explicit and fine-grained editing; they either fail to generate additional objects on given portraits or cause alterations to portraits (e.g., identity shift) when generating accessories. This restriction presents a noteworthy obstacle as people typically seek to create charming appearances with diverse and fashionable accessories in the virtual universe. Our approach provides an effective solution to this less-addressed issue. We further introduce the Scribble2Accessories module, enabling Pomo3D to create 3D accessories from user-drawn accessory scribble maps. Moreover, we design a bias-conscious mapper to mitigate biased associations present in real-world datasets. In addition to object-level manipulation above, Pomo3D also offers extensive editing options on portraits, including global or local editing of geometry and texture and avatar stylization, elevating 3D editing of neural portraits to a more comprehensive level.