Context Tricks for Cheap Semantic Segmentation

TL;DR

This paper introduces two simple context-based strategies, Decorrelated Semantic Texton Forests and Context Sensitive Image Level Prior, to improve semantic segmentation accuracy while maintaining efficiency and scalability across datasets.

Contribution

The paper proposes novel, easy-to-implement modifications that leverage context to enhance semantic segmentation performance and scalability, validated on standard benchmarks.

Findings

Superior segmentation results on MSRC-21 and PascalVOC-2010

Minimal increase in test-time computational cost

Effective training on small datasets with simple modifications

Abstract

Accurate semantic labeling of image pixels is difficult because intra-class variability is often greater than inter-class variability. In turn, fast semantic segmentation is hard because accurate models are usually too complicated to also run quickly at test-time. Our experience with building and running semantic segmentation systems has also shown a reasonably obvious bottleneck on model complexity, imposed by small training datasets. We therefore propose two simple complementary strategies that leverage context to give better semantic segmentation, while scaling up or down to train on different-sized datasets. As easy modifications for existing semantic segmentation algorithms, we introduce Decorrelated Semantic Texton Forests, and the Context Sensitive Image Level Prior. The proposed modifications are tested using a Semantic Texton Forest (STF) system, and the modifications are validated on two standard benchmark datasets, MSRC-21 and PascalVOC-2010. In Python based comparisons, our system is insignificantly slower than STF at test-time, yet produces superior semantic segmentations overall, with just push-button training.