LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstract
Xingzhe He, Bastian Wandt, Helge Rhodin

TL;DR
LatentKeypointGAN introduces a self-supervised two-stage GAN that enables detailed control over image content through interpretable latent keypoints, allowing flexible editing and combination of image parts across diverse domains.
Contribution
It presents a novel self-supervised framework that disentangles spatial and appearance factors using latent keypoints, facilitating controllable image generation without labels.
Findings
Provides an interpretable latent space for image editing.
Enables re-arrangement and combination of image parts.
Works across diverse application domains.
Abstract
Generative adversarial networks (GANs) can now generate photo-realistic images. However, how to best control the image content remains an open challenge. We introduce LatentKeypointGAN, a two-stage GAN internally conditioned on a set of keypoints and associated appearance embeddings providing control of the position and style of the generated objects and their respective parts. A major difficulty that we address is disentangling the image into spatial and appearance factors with little domain knowledge and supervision signals. We demonstrate in a user study and quantitative experiments that LatentKeypointGAN provides an interpretable latent space that can be used to re-arrange the generated images by re-positioning and exchanging keypoint embeddings, such as generating portraits by combining the eyes, and mouth from different images. Notably, our method does not require labels as it is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Advanced Vision and Imaging · 3D Surveying and Cultural Heritage
