Illiterate DALL-E Learns to Compose
Gautam Singh, Fei Deng, Sungjin Ahn

TL;DR
This paper introduces SLATE, a novel slot-based autoencoding model that learns object-centric representations enabling zero-shot image generation without text prompts, combining systematic generalization with simplicity.
Contribution
SLATE is a new architecture that merges object-centric learning with zero-shot image generation, eliminating the need for text prompts and using a novel decoder conditioned on slots.
Findings
Significant improvement in zero-shot image generation quality.
Qualitatively comparable or better slot-attention structures.
Effective in both in-distribution and out-of-distribution scenarios.
Abstract
Although DALL-E has shown an impressive ability of composition-based systematic generalization in image generation, it requires the dataset of text-image pairs and the compositionality is provided by the text. In contrast, object-centric representation models like the Slot Attention model learn composable representations without the text prompt. However, unlike DALL-E its ability to systematically generalize for zero-shot generation is significantly limited. In this paper, we propose a simple but novel slot-based autoencoding architecture, called SLATE, for combining the best of both worlds: learning object-centric representations that allows systematic generalization in zero-shot image generation without text. As such, this model can also be seen as an illiterate DALL-E model. Unlike the pixel-mixture decoders of existing object-centric representation models, we propose to use the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsMultimodal Machine Learning Applications · Domain Adaptation and Few-Shot Learning · Generative Adversarial Networks and Image Synthesis
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Cosine Annealing · Byte Pair Encoding · Linear Warmup With Cosine Annealing · Weight Decay · Attention Dropout · Refunds@Expedia|||How do I get a full refund from Expedia? · Adam
