Non-parametric spatially constrained local prior for scene parsing on real-world data

TL;DR

This paper introduces a non-parametric spatially constrained local prior (SCLP) method for scene parsing that leverages retrieved similar images to incorporate object co-occurrence and spatial context, improving accuracy on real-world datasets.

Contribution

The paper proposes a novel non-parametric SCLP approach that captures both long- and short-range contextual information for scene parsing in realistic data.

Findings

Achieves top performance on SIFT Flow dataset

Effective integration with traditional visual features

Outperforms several state-of-the-art methods

Abstract

Scene parsing aims to recognize the object category of every pixel in scene images, and it plays a central role in image content understanding and computer vision applications. However, accurate scene parsing from unconstrained real-world data is still a challenging task. In this paper, we present the non-parametric Spatially Constrained Local Prior (SCLP) for scene parsing on realistic data. For a given query image, the non-parametric SCLP is learnt by first retrieving a subset of most similar training images to the query image and then collecting prior information about object co-occurrence statistics between spatial image blocks and between adjacent superpixels from the retrieved subset. The SCLP is powerful in capturing both long- and short-range context about inter-object correlations in the query image and can be effectively integrated with traditional visual features to refine the classification results. Our experiments on the SIFT Flow and PASCAL-Context benchmark datasets show that the non-parametric SCLP used in conjunction with superpixel-level visual features achieves one of the top performance compared with state-of-the-art approaches.