Superpixel Sampling Networks

TL;DR

The paper introduces a differentiable superpixel sampling network that can be trained end-to-end, improving superpixel quality and integration into deep learning pipelines for various vision tasks.

Contribution

A novel differentiable superpixel sampling model that enables end-to-end training and task-specific superpixel learning within deep neural networks.

Findings

Outperforms existing superpixel algorithms on segmentation benchmarks.

Can learn superpixels tailored for specific tasks.

Easily integrates into downstream deep networks, enhancing performance.

Abstract

Superpixels provide an efficient low/mid-level representation of image data, which greatly reduces the number of image primitives for subsequent vision tasks. Existing superpixel algorithms are not differentiable, making them difficult to integrate into otherwise end-to-end trainable deep neural networks. We develop a new differentiable model for superpixel sampling that leverages deep networks for learning superpixel segmentation. The resulting "Superpixel Sampling Network" (SSN) is end-to-end trainable, which allows learning task-specific superpixels with flexible loss functions and has fast runtime. Extensive experimental analysis indicates that SSNs not only outperform existing superpixel algorithms on traditional segmentation benchmarks, but can also learn superpixels for other tasks. In addition, SSNs can be easily integrated into downstream deep networks resulting in performance improvements.