Learning to Structure an Image with Few Colors and Beyond

TL;DR

This paper introduces ColorCNN+ a neural network that learns to structure images with limited colors, improving recognition and visual fidelity, and explores the importance of color and structure in neural network recognition.

Contribution

We propose ColorCNN+ a novel color quantization network that supports multiple color spaces and learns to cluster colors, enhancing recognition accuracy and visual fidelity in limited color scenarios.

Findings

ColorCNN+ achieves competitive recognition accuracy.

It preserves key structures and accurate colors.

Supports multiple color space configurations.

Abstract

Color and structure are the two pillars that combine to give an image its meaning. Interested in critical structures for neural network recognition, we isolate the influence of colors by limiting the color space to just a few bits, and find structures that enable network recognition under such constraints. To this end, we propose a color quantization network, ColorCNN, which learns to structure an image in limited color spaces by minimizing the classification loss. Building upon the architecture and insights of ColorCNN, we introduce ColorCNN+, which supports multiple color space size configurations, and addresses the previous issues of poor recognition accuracy and undesirable visual fidelity under large color spaces. Via a novel imitation learning approach, ColorCNN+ learns to cluster colors like traditional color quantization methods. This reduces overfitting and helps both visual fidelity and recognition accuracy under large color spaces. Experiments verify that ColorCNN+ achieves very competitive results under most circumstances, preserving both key structures for network recognition and visual fidelity with accurate colors. We further discuss differences between key structures and accurate colors, and their specific contributions to network recognition. For potential applications, we show that ColorCNNs can be used as image compression methods for network recognition.