HERS Superpixels: Deep Affinity Learning for Hierarchical Entropy Rate Segmentation

TL;DR

This paper introduces a fast, two-stage superpixel segmentation framework combining deep affinity learning with a hierarchical entropy rate method, enabling near real-time, adaptive superpixel generation for computer vision tasks.

Contribution

It presents a novel two-stage framework with a deep affinity network and a hierarchical segmentation method, achieving efficient and adaptive superpixel segmentation in real-time.

Findings

Outperforms state-of-the-art superpixel methods in accuracy.

Operates in near real-time suitable for practical applications.

Provides highly adaptive superpixels through hierarchical structure.

Abstract

Superpixels serve as a powerful preprocessing tool in numerous computer vision tasks. By using superpixel representation, the number of image primitives can be largely reduced by orders of magnitudes. With the rise of deep learning in recent years, a few works have attempted to feed deeply learned features / graphs into existing classical superpixel techniques. However, none of them are able to produce superpixels in near real-time, which is crucial to the applicability of superpixels in practice. In this work, we propose a two-stage graph-based framework for superpixel segmentation. In the first stage, we introduce an efficient Deep Affinity Learning (DAL) network that learns pairwise pixel affinities by aggregating multi-scale information. In the second stage, we propose a highly efficient superpixel method called Hierarchical Entropy Rate Segmentation (HERS). Using the learned affinities from the first stage, HERS builds a hierarchical tree structure that can produce any number of highly adaptive superpixels instantaneously. We demonstrate, through visual and numerical experiments, the effectiveness and efficiency of our method compared to various state-of-the-art superpixel methods.