LevelSet R-CNN: A Deep Variational Method for Instance Segmentation

TL;DR

LevelSet R-CNN integrates deep learning with variational segmentation to produce precise instance masks, overcoming limitations of low-resolution outputs and manual hyperparameter tuning in existing models.

Contribution

It introduces a novel end-to-end framework combining feature extraction with a variational segmentation approach for improved instance mask accuracy.

Findings

Effective on COCO dataset

Outperforms traditional variational methods

Produces more precise segmentation masks

Abstract

Obtaining precise instance segmentation masks is of high importance in many modern applications such as robotic manipulation and autonomous driving. Currently, many state of the art models are based on the Mask R-CNN framework which, while very powerful, outputs masks at low resolutions which could result in imprecise boundaries. On the other hand, classic variational methods for segmentation impose desirable global and local data and geometry constraints on the masks by optimizing an energy functional. While mathematically elegant, their direct dependence on good initialization, non-robust image cues and manual setting of hyperparameters renders them unsuitable for modern applications. We propose LevelSet R-CNN, which combines the best of both worlds by obtaining powerful feature representations that are combined in an end-to-end manner with a variational segmentation framework. We demonstrate the effectiveness of our approach on COCO and Cityscapes datasets.