Conditional Reconstruction for Open-set Semantic Segmentation

TL;DR

The paper introduces CoReSeg, a class conditional reconstruction method for open-set semantic segmentation that improves boundary accuracy and outperforms existing methods on multiple datasets.

Contribution

It presents a novel class conditional reconstruction approach for open-set segmentation, enhancing semantic consistency and boundary accuracy.

Findings

Outperforms state-of-the-art on Vaihin-gen and Potsdam datasets.

Produces cleaner segmentation maps with better object boundary fit.

Competitive results on Houston 2018 IEEE dataset.

Abstract

Open set segmentation is a relatively new and unexploredtask, with just a handful of methods proposed to model suchtasks.We propose a novel method called CoReSeg thattackles the issue using class conditional reconstruction ofthe input images according to their pixelwise mask. Ourmethod conditions each input pixel to all known classes,expecting higher errors for pixels of unknown classes. Itwas observed that the proposed method produces better se-mantic consistency in its predictions, resulting in cleanersegmentation maps that better fit object boundaries. CoRe-Seg outperforms state-of-the-art methods on the Vaihin-gen and Potsdam ISPRS datasets, while also being com-petitive on the Houston 2018 IEEE GRSS Data Fusiondataset. Official implementation for CoReSeg is availableat:https://github.com/iannunes/CoReSeg.