Using Image Priors to Improve Scene Understanding

TL;DR

This paper introduces a prior fusion network that leverages temporal scene information to significantly improve semantic segmentation accuracy in autonomous driving scenarios, using fewer parameters than traditional models.

Contribution

The paper presents a novel prior fusion network that effectively incorporates temporal scene priors to enhance semantic segmentation performance in sequential driving data.

Findings

Prior fusion improves dynamic class accuracy from 69.1% to 73.3%.

Static class accuracy improves from 88.2% to 89.1%.

Achieves similar accuracy to FCN-8 with 5x fewer parameters.

Abstract

Semantic segmentation algorithms that can robustly segment objects across multiple camera viewpoints are crucial for assuring navigation and safety in emerging applications such as autonomous driving. Existing algorithms treat each image in isolation, but autonomous vehicles often revisit the same locations or maintain information from the immediate past. We propose a simple yet effective method for leveraging these image priors to improve semantic segmentation of images from sequential driving datasets. We examine several methods to fuse these temporal scene priors, and introduce a prior fusion network that is able to learn how to transfer this information. The prior fusion model improves the accuracy over the non-prior baseline from 69.1% to 73.3% for dynamic classes, and from 88.2% to 89.1% for static classes. Compared to models such as FCN-8, our prior method achieves the same accuracy with 5 times fewer parameters. We used a simple encoder decoder backbone, but this general prior fusion method could be applied to more complex semantic segmentation backbones. We also discuss how structured representations of scenes in the form of a scene graph could be leveraged as priors to further improve scene understanding.