What Else Can Fool Deep Learning? Addressing Color Constancy Errors on Deep Neural Network Performance

TL;DR

This paper investigates how color cast errors from incorrect white balance in images impair deep neural network performance and proposes augmentation and pre-processing methods to mitigate these effects, improving accuracy on multiple datasets.

Contribution

It introduces a novel augmentation technique to simulate white balance errors and evaluates pre-processing methods to enhance DNN robustness against color constancy errors.

Findings

Augmentation improves model robustness to white balance errors.

Pre-processing with WB correction reduces error impact.

Significant accuracy gains on CIFAR-10, CIFAR-100, and ADE20K.

Abstract

There is active research targeting local image manipulations that can fool deep neural networks (DNNs) into producing incorrect results. This paper examines a type of global image manipulation that can produce similar adverse effects. Specifically, we explore how strong color casts caused by incorrectly applied computational color constancy - referred to as white balance (WB) in photography - negatively impact the performance of DNNs targeting image segmentation and classification. In addition, we discuss how existing image augmentation methods used to improve the robustness of DNNs are not well suited for modeling WB errors. To address this problem, a novel augmentation method is proposed that can emulate accurate color constancy degradation. We also explore pre-processing training and testing images with a recent WB correction algorithm to reduce the effects of incorrectly white-balanced images. We examine both augmentation and pre-processing strategies on different datasets and demonstrate notable improvements on the CIFAR-10, CIFAR-100, and ADE20K datasets.