Learning Color Equivariant Representations

TL;DR

This paper introduces color-equivariant group convolutional neural networks that handle hue, saturation, and luminance variations, improving generalization and sample efficiency in visual tasks.

Contribution

We develop a novel lifting layer for GCNNs that achieves robust color equivariance, extending it beyond hue to saturation and luminance shifts.

Findings

Our networks outperform baselines on synthetic and real datasets.

The lifting layer reduces equivariance error by over three orders of magnitude.

The approach improves out-of-distribution generalization and sample efficiency.

Abstract

In this paper, we introduce group convolutional neural networks (GCNNs) equivariant to color variation. GCNNs have been designed for a variety of geometric transformations from 2D and 3D rotation groups, to semi-groups such as scale. Despite the improved interpretability, accuracy and generalizability of these architectures, GCNNs have seen limited application in the context of perceptual quantities. Notably, the recent CEConv network uses a GCNN to achieve equivariance to hue transformations by convolving input images with a hue rotated RGB filter. However, this approach leads to invalid RGB values which break equivariance and degrade performance. We resolve these issues with a lifting layer that transforms the input image directly, thereby circumventing the issue of invalid RGB values and improving equivariance error by over three orders of magnitude. Moreover, we extend the notion of color equivariance to include equivariance to saturation and luminance shift. Our hue-, saturation-, luminance- and color-equivariant networks achieve strong generalization to out-of-distribution perceptual variations and improved sample efficiency over conventional architectures. We demonstrate the utility of our approach on synthetic and real world datasets where we consistently outperform competitive baselines.