BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks

TL;DR

BiSeg is a fully convolutional network framework that performs simultaneous semantic and instance segmentation by modeling instance segmentation as Bayesian inference, achieving state-of-the-art results on PASCAL VOC.

Contribution

It introduces a Bayesian inference approach for instance segmentation combined with semantic segmentation within a fully convolutional network architecture.

Findings

Achieves state-of-the-art instance segmentation accuracy on PASCAL VOC.

Proposes a fusion of multi-scale, multi-mode score maps for robust inference.

Demonstrates a fully convolutional, end-to-end solution for joint segmentation tasks.

Abstract

We present a simple and effective framework for simultaneous semantic segmentation and instance segmentation with Fully Convolutional Networks (FCNs). The method, called BiSeg, predicts instance segmentation as a posterior in Bayesian inference, where semantic segmentation is used as a prior. We extend the idea of position-sensitive score maps used in recent methods to a fusion of multiple score maps at different scales and partition modes, and adopt it as a robust likelihood for instance segmentation inference. As both Bayesian inference and map fusion are performed per pixel, BiSeg is a fully convolutional end-to-end solution that inherits all the advantages of FCNs. We demonstrate state-of-the-art instance segmentation accuracy on PASCAL VOC.