Learning to Cluster for Proposal-Free Instance Segmentation

TL;DR

This paper introduces a novel end-to-end deep learning approach for proposal-free instance segmentation by training a neural network to perform pixel clustering based on pairwise pixel relationships, achieving strong results on Cityscapes.

Contribution

It presents a new learning objective for pixel clustering in instance segmentation, incorporating graph coloring ideas to handle unlimited instances, and demonstrates competitive performance.

Findings

Achieved strong performance on Cityscapes dataset.

Won second place in 2017 CVPR Autonomous Driving Challenge lane detection.

Top performer without external data.

Abstract

This work proposed a novel learning objective to train a deep neural network to perform end-to-end image pixel clustering. We applied the approach to instance segmentation, which is at the intersection of image semantic segmentation and object detection. We utilize the most fundamental property of instance labeling -- the pairwise relationship between pixels -- as the supervision to formulate the learning objective, then apply it to train a fully convolutional network (FCN) for learning to perform pixel-wise clustering. The resulting clusters can be used as the instance labeling directly. To support labeling of an unlimited number of instance, we further formulate ideas from graph coloring theory into the proposed learning objective. The evaluation on the Cityscapes dataset demonstrates strong performance and therefore proof of the concept. Moreover, our approach won the second place in the lane detection competition of 2017 CVPR Autonomous Driving Challenge, and was the top performer without using external data.