Learning to Generate Compositional Color Descriptions

TL;DR

This paper introduces a neural network model that effectively generates detailed and compositional color descriptions, outperforming previous methods and capturing complex language features like modifiers and non-convex denotations.

Contribution

The paper presents a novel neural approach using Fourier-transformed color representations for generating rich, compositional color descriptions, advancing grounded language generation.

Findings

Model outperforms previous work on color description generation

Accurately produces complex descriptors like 'greenish' and 'faded teal'

Captures non-convex and compositional language features

Abstract

The production of color language is essential for grounded language generation. Color descriptions have many challenging properties: they can be vague, compositionally complex, and denotationally rich. We present an effective approach to generating color descriptions using recurrent neural networks and a Fourier-transformed color representation. Our model outperforms previous work on a conditional language modeling task over a large corpus of naturalistic color descriptions. In addition, probing the model's output reveals that it can accurately produce not only basic color terms but also descriptors with non-convex denotations ("greenish"), bare modifiers ("bright", "dull"), and compositional phrases ("faded teal") not seen in training.