A Bag of Words Approach for Semantic Segmentation of Monitored Scenes

TL;DR

This paper introduces a semantic segmentation technique for outdoor surveillance scenes that combines color segmentation with SIFT keypoints and a bag of words model, resulting in more human-like and compact scene understanding.

Contribution

The paper presents a novel combination of color segmentation and SIFT-based bag of words for improved semantic segmentation of outdoor scenes.

Findings

More compact scene representations achieved.

Segmentation aligns better with human perception.

Validated on a public dataset with positive results.

Abstract

This paper proposes a semantic segmentation method for outdoor scenes captured by a surveillance camera. Our algorithm classifies each perceptually homogenous region as one of the predefined classes learned from a collection of manually labelled images. The proposed approach combines two different types of information. First, color segmentation is performed to divide the scene into perceptually similar regions. Then, the second step is based on SIFT keypoints and uses the bag of words representation of the regions for the classification. The prediction is done using a Na\"ive Bayesian Network as a generative classifier. Compared to existing techniques, our method provides more compact representations of scene contents and the segmentation result is more consistent with human perception due to the combination of the color information with the image keypoints. The experiments conducted on a publicly available data set demonstrate the validity of the proposed method.