Context Prior for Scene Segmentation

TL;DR

This paper introduces a supervised Context Prior mechanism with Affinity Loss to distinguish intra-class and inter-class dependencies, improving scene segmentation accuracy.

Contribution

It proposes a novel Context Prior Layer supervised by Affinity Loss, enabling selective capture of intra- and inter-class contextual dependencies within CNNs.

Findings

Achieves 46.3% mIoU on ADE20K

Outperforms state-of-the-art segmentation methods

Demonstrates effective separation of intra- and inter-class contexts

Abstract

Recent works have widely explored the contextual dependencies to achieve more accurate segmentation results. However, most approaches rarely distinguish different types of contextual dependencies, which may pollute the scene understanding. In this work, we directly supervise the feature aggregation to distinguish the intra-class and inter-class context clearly. Specifically, we develop a Context Prior with the supervision of the Affinity Loss. Given an input image and corresponding ground truth, Affinity Loss constructs an ideal affinity map to supervise the learning of Context Prior. The learned Context Prior extracts the pixels belonging to the same category, while the reversed prior focuses on the pixels of different classes. Embedded into a conventional deep CNN, the proposed Context Prior Layer can selectively capture the intra-class and inter-class contextual dependencies, leading to robust feature representation. To validate the effectiveness, we design an effective Context Prior Network (CPNet). Extensive quantitative and qualitative evaluations demonstrate that the proposed model performs favorably against state-of-the-art semantic segmentation approaches. More specifically, our algorithm achieves 46.3% mIoU on ADE20K, 53.9% mIoU on PASCAL-Context, and 81.3% mIoU on Cityscapes. Code is available at https://git.io/ContextPrior.