Contextrast: Contextual Contrastive Learning for Semantic Segmentation

TL;DR

Contextrast introduces a contrastive learning framework that captures local and global contexts and their relationships, improving semantic segmentation accuracy by effectively handling boundary regions without extra inference cost.

Contribution

The paper proposes Contextrast, a novel contrastive learning method with boundary-aware sampling that enhances segmentation by leveraging multi-scale features and boundary information.

Findings

Outperforms state-of-the-art contrastive methods on multiple datasets

Improves boundary region segmentation accuracy

No additional inference cost

Abstract

Despite great improvements in semantic segmentation, challenges persist because of the lack of local/global contexts and the relationship between them. In this paper, we propose Contextrast, a contrastive learning-based semantic segmentation method that allows to capture local/global contexts and comprehend their relationships. Our proposed method comprises two parts: a) contextual contrastive learning (CCL) and b) boundary-aware negative (BANE) sampling. Contextual contrastive learning obtains local/global context from multi-scale feature aggregation and inter/intra-relationship of features for better discrimination capabilities. Meanwhile, BANE sampling selects embedding features along the boundaries of incorrectly predicted regions to employ them as harder negative samples on our contrastive learning, resolving segmentation issues along the boundary region by exploiting fine-grained details. We demonstrate that our Contextrast substantially enhances the performance of semantic segmentation networks, outperforming state-of-the-art contrastive learning approaches on diverse public datasets, e.g. Cityscapes, CamVid, PASCAL-C, COCO-Stuff, and ADE20K, without an increase in computational cost during inference.