StylePart: Image-based Shape Part Manipulation

TL;DR

StylePart introduces a novel framework that enables intuitive, image-based manipulation of man-made shapes by bridging image and 3D shape latent spaces, allowing for direct editing of shape parts.

Contribution

The paper proposes a shape-consistent latent mapping that connects image and 3D shape spaces, facilitating intuitive shape part manipulation directly from images.

Findings

Effective shape part replacement and resizing demonstrated

Viewpoint manipulation achieved with high fidelity

Extensive ablation studies validate the method's robustness

Abstract

Due to a lack of image-based "part controllers", shape manipulation of man-made shape images, such as resizing the backrest of a chair or replacing a cup handle is not intuitive. To tackle this problem, we present StylePart, a framework that enables direct shape manipulation of an image by leveraging generative models of both images and 3D shapes. Our key contribution is a shape-consistent latent mapping function that connects the image generative latent space and the 3D man-made shape attribute latent space. Our method "forwardly maps" the image content to its corresponding 3D shape attributes, where the shape part can be easily manipulated. The attribute codes of the manipulated 3D shape are then "backwardly mapped" to the image latent code to obtain the final manipulated image. We demonstrate our approach through various manipulation tasks, including part replacement, part resizing, and viewpoint manipulation, and evaluate its effectiveness through extensive ablation studies.