Generating Compositional Color Representations from Text
Paridhi Maheshwari, Nihal Jain, Praneetha Vaddamanu, Dhananjay Raut,, Shraiysh Vaishay, Vishwa Vinay

TL;DR
This paper introduces a generative adversarial network that creates color representations from text phrases, especially attribute-object pairs, and demonstrates its effectiveness in image retrieval and classification tasks.
Contribution
It presents a novel GAN-based approach for compositional color generation from text, along with a new dataset creation pipeline and extensive ablation studies.
Findings
GAN achieves lower Frechet Inception Distance than discriminative models.
Generated color profiles improve image retrieval accuracy.
Method generalizes to other visual dimensions involving composition.
Abstract
We consider the cross-modal task of producing color representations for text phrases. Motivated by the fact that a significant fraction of user queries on an image search engine follow an (attribute, object) structure, we propose a generative adversarial network that generates color profiles for such bigrams. We design our pipeline to learn composition - the ability to combine seen attributes and objects to unseen pairs. We propose a novel dataset curation pipeline from existing public sources. We describe how a set of phrases of interest can be compiled using a graph propagation technique, and then mapped to images. While this dataset is specialized for our investigations on color, the method can be extended to other visual dimensions where composition is of interest. We provide detailed ablation studies that test the behavior of our GAN architecture with loss functions from the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Retrieval and Classification Techniques · Multimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques
MethodsTest · Contrastive Learning
