Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation

TL;DR

This paper introduces a novel semantic segmentation model that combines global and local features using Gaussian Mixture Models, improving boundary precision and overall accuracy on Cityscapes and Synthia datasets.

Contribution

The proposed model uniquely integrates Gaussian Mixture Models into deep segmentation networks to enhance feature representation and boundary accuracy.

Findings

Improved boundary segmentation accuracy.

Enhanced global and local feature integration.

Better performance on Cityscapes and Synthia datasets.

Abstract

The semantic segmentation task aims at dense classification at the pixel-wise level. Deep models exhibited progress in tackling this task. However, one remaining problem with these approaches is the loss of spatial precision, often produced at the segmented objects' boundaries. Our proposed model addresses this problem by providing an internal structure for the feature representations while extracting a global representation that supports the former. To fit the internal structure, during training, we predict a Gaussian Mixture Model from the data, which, merged with the skip connections and the decoding stage, helps avoid wrong inductive biases. Furthermore, our results show that we can improve semantic segmentation by providing both learning representations (global and local) with a clustering behavior and combining them. Finally, we present results demonstrating our advances in Cityscapes and Synthia datasets.