Grounding learning of modifier dynamics: An application to color naming

TL;DR

This paper introduces a model for understanding modified color expressions by learning complex transformations in RGB and HSV spaces, improving accuracy over previous additive models.

Contribution

It presents a novel ensemble approach that selects the optimal color space for different modifiers, enhancing the modeling of color adjective semantics.

Findings

Significant improvements over baseline models

HSV space better models certain adjectives

Ensemble model adapts to different color modifiers

Abstract

Grounding is crucial for natural language understanding. An important subtask is to understand modified color expressions, such as 'dirty blue'. We present a model of color modifiers that, compared with previous additive models in RGB space, learns more complex transformations. In addition, we present a model that operates in the HSV color space. We show that certain adjectives are better modeled in that space. To account for all modifiers, we train a hard ensemble model that selects a color space depending on the modifier color pair. Experimental results show significant and consistent improvements compared to the state-of-the-art baseline model.