Local Context Normalization: Revisiting Local Normalization

TL;DR

This paper introduces Local Context Normalization (LCN), a normalization method that considers local spatial context and group information, improving deep neural network performance in vision tasks without batch size dependence.

Contribution

The paper proposes LCN, a novel normalization layer that normalizes features based on local spatial windows and group info, with an efficient algorithm for arbitrary window sizes.

Findings

LCN outperforms BN, GN, IN, and LN in object detection, segmentation tasks.

LCN maintains performance regardless of batch size.

LCN facilitates transfer learning in vision applications.

Abstract

Normalization layers have been shown to improve convergence in deep neural networks, and even add useful inductive biases. In many vision applications the local spatial context of the features is important, but most common normalization schemes including Group Normalization (GN), Instance Normalization (IN), and Layer Normalization (LN) normalize over the entire spatial dimension of a feature. This can wash out important signals and degrade performance. For example, in applications that use satellite imagery, input images can be arbitrarily large; consequently, it is nonsensical to normalize over the entire area. Positional Normalization (PN), on the other hand, only normalizes over a single spatial position at a time. A natural compromise is to normalize features by local context, while also taking into account group level information. In this paper, we propose Local Context Normalization (LCN): a normalization layer where every feature is normalized based on a window around it and the filters in its group. We propose an algorithmic solution to make LCN efficient for arbitrary window sizes, even if every point in the image has a unique window. LCN outperforms its Batch Normalization (BN), GN, IN, and LN counterparts for object detection, semantic segmentation, and instance segmentation applications in several benchmark datasets, while keeping performance independent of the batch size and facilitating transfer learning.