Sample what you cant compress
Vighnesh Birodkar, Gabriel Barcik, James Lyon, Sergey Ioffe, David Minnen, Joshua V. Dillon

TL;DR
This paper introduces SWYCC, a novel autoencoder framework combining diffusion models with representation learning, achieving higher compression, improved reconstruction quality, and more flexible generation than GAN-based methods.
Contribution
It is the first to jointly learn a continuous encoder and decoder using diffusion-based loss, demonstrating superior performance and easier tuning compared to existing GAN-based autoencoders.
Findings
Better reconstruction quality than GAN-based autoencoders
Easier to tune than adversarial methods
Representation more amenable to latent diffusion modeling
Abstract
For learned image representations, basic autoencoders often produce blurry results. Reconstruction quality can be improved by incorporating additional penalties such as adversarial (GAN) and perceptual losses. Arguably, these approaches lack a principled interpretation. Concurrently, in generative settings diffusion has demonstrated a remarkable ability to create crisp, high quality results and has solid theoretical underpinnings (from variational inference to direct study as the Fisher Divergence). Our work combines autoencoder representation learning with diffusion and is, to our knowledge, the first to demonstrate jointly learning a continuous encoder and decoder under a diffusion-based loss and showing that it can lead to higher compression and better generation. We demonstrate that this approach yields better reconstruction quality as compared to GAN-based autoencoders while being…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Advanced Image Processing Techniques · Face recognition and analysis
MethodsDiffusion · Latent Diffusion Model · Variational Inference
