Dense Self-Supervised Learning for Medical Image Segmentation

TL;DR

Pix2Rep introduces a self-supervised learning method that learns pixel-level representations from unlabeled images, significantly reducing annotation needs and improving medical image segmentation performance, especially in cardiac MRI tasks.

Contribution

The paper presents Pix2Rep, a novel pixel-level contrastive SSL framework enforcing equivariance, which enhances segmentation accuracy and reduces annotation effort in medical imaging.

Findings

Achieves 30-31% DICE improvement over baseline.

Reduces annotation burden by 5-fold.

Enhances segmentation performance with Pix2Rep and Barlow Twins integration.

Abstract

Deep learning has revolutionized medical image segmentation, but it relies heavily on high-quality annotations. The time, cost and expertise required to label images at the pixel-level for each new task has slowed down widespread adoption of the paradigm. We propose Pix2Rep, a self-supervised learning (SSL) approach for few-shot segmentation, that reduces the manual annotation burden by learning powerful pixel-level representations directly from unlabeled images. Pix2Rep is a novel pixel-level loss and pre-training paradigm for contrastive SSL on whole images. It is applied to generic encoder-decoder deep learning backbones (e.g., U-Net). Whereas most SSL methods enforce invariance of the learned image-level representations under intensity and spatial image augmentations, Pix2Rep enforces equivariance of the pixel-level representations. We demonstrate the framework on a task of cardiac MRI segmentation. Results show improved performance compared to existing semi- and self-supervised approaches; and a 5-fold reduction in the annotation burden for equivalent performance versus a fully supervised U-Net baseline. This includes a 30% (resp. 31%) DICE improvement for one-shot segmentation under linear-probing (resp. fine-tuning). Finally, we also integrate the novel Pix2Rep concept with the Barlow Twins non-contrastive SSL, which leads to even better segmentation performance.