Information-guided pixel augmentation for pixel-wise contrastive learning

TL;DR

This paper introduces an information-guided pixel augmentation strategy for pixel-wise contrastive learning, improving feature representations for pixel-level tasks like medical landmark detection by classifying pixels and applying tailored augmentations.

Contribution

It is the first to propose a pixel-level augmentation method guided by pixel informativeness, enhancing unsupervised pixel-wise contrastive learning performance.

Findings

Outperforms other methods in unsupervised local feature matching

Improves downstream task performance for supervised models

Classifies pixels into three informativeness categories for tailored augmentation

Abstract

Contrastive learning (CL) is a form of self-supervised learning and has been widely used for various tasks. Different from widely studied instance-level contrastive learning, pixel-wise contrastive learning mainly helps with pixel-wise tasks such as medical landmark detection. The counterpart to an instance in instance-level CL is a pixel, along with its neighboring context, in pixel-wise CL. Aiming to build better feature representation, there is a vast literature about designing instance augmentation strategies for instance-level CL; but there is little similar work on pixel augmentation for pixel-wise CL with a pixel granularity. In this paper, we attempt to bridge this gap. We first classify a pixel into three categories, namely low-, medium-, and high-informative, based on the information quantity the pixel contains. Inspired by the ``InfoMin" principle, we then design separate augmentation strategies for each category in terms of augmentation intensity and sampling ratio. Extensive experiments validate that our information-guided pixel augmentation strategy succeeds in encoding more discriminative representations and surpassing other competitive approaches in unsupervised local feature matching. Furthermore, our pretrained model improves the performance of both one-shot and fully supervised models. To the best of our knowledge, we are the first to propose a pixel augmentation method with a pixel granularity for enhancing unsupervised pixel-wise contrastive learning.