Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations

TL;DR

This paper analyzes the properties of dense contrastive representations, revealing their correlation with performance and introducing a new metric to quantify this relationship across different tasks.

Contribution

It provides the first detailed analysis of dense contrastive learning properties using alignment and uniformity, and introduces a scalar metric to measure their correlation with performance.

Findings

Positive pair construction is crucial for dense contrastive learning.

The new metric effectively correlates dense representation quality with downstream performance.

Dense contrastive representations show varying correlation patterns across datasets and tasks.

Abstract

Recently, dense contrastive learning has shown superior performance on dense prediction tasks compared to instance-level contrastive learning. Despite its supremacy, the properties of dense contrastive representations have not yet been carefully studied. Therefore, we analyze the theoretical ideas of dense contrastive learning using a standard CNN and straightforward feature matching scheme rather than propose a new complex method. Inspired by the analysis of the properties of instance-level contrastive representations through the lens of alignment and uniformity on the hypersphere, we employ and extend the same lens for the dense contrastive representations to analyze their underexplored properties. We discover the core principle in constructing a positive pair of dense features and empirically proved its validity. Also, we introduces a new scalar metric that summarizes the correlation between alignment-and-uniformity and downstream performance. Using this metric, we study various facets of densely learned contrastive representations such as how the correlation changes over single- and multi-object datasets or linear evaluation and dense prediction tasks. The source code is publicly available at: https://github.com/SuperSupermoon/DenseCL-analysis