Adaptative Context Normalization: A Boost for Deep Learning in Image Processing

TL;DR

This paper introduces Adaptative Context Normalization (ACN), a supervised normalization method that groups data by context for improved speed, convergence, and performance in deep image processing models.

Contribution

ACN offers a novel supervised normalization approach that leverages context grouping, outperforming Batch Normalization and Mixture Normalization in image processing tasks.

Findings

ACN achieves faster convergence than BN and MN.

ACN demonstrates superior performance in image processing benchmarks.

ACN effectively models local data characteristics through context grouping.

Abstract

Deep Neural network learning for image processing faces major challenges related to changes in distribution across layers, which disrupt model convergence and performance. Activation normalization methods, such as Batch Normalization (BN), have revolutionized this field, but they rely on the simplified assumption that data distribution can be modelled by a single Gaussian distribution. To overcome these limitations, Mixture Normalization (MN) introduced an approach based on a Gaussian Mixture Model (GMM), assuming multiple components to model the data. However, this method entails substantial computational requirements associated with the use of Expectation-Maximization algorithm to estimate parameters of each Gaussian components. To address this issue, we introduce Adaptative Context Normalization (ACN), a novel supervised approach that introduces the concept of "context", which groups together a set of data with similar characteristics. Data belonging to the same context are normalized using the same parameters, enabling local representation based on contexts. For each context, the normalized parameters, as the model weights are learned during the backpropagation phase. ACN not only ensures speed, convergence, and superior performance compared to BN and MN but also presents a fresh perspective that underscores its particular efficacy in the field of image processing.