Generating Superpixels for High-resolution Images with Decoupled Patch Calibration

TL;DR

This paper introduces PCNet, a novel deep learning approach for high-resolution superpixel segmentation that reduces memory and computation costs while maintaining accuracy, by decoupling patch calibration from the main network.

Contribution

It proposes a decoupled patch calibration branch within a deep learning framework to efficiently process high-resolution images for superpixel segmentation.

Findings

Outperforms state-of-the-art methods in accuracy

Increases resolution processing from 3K to 5K on GPUs

Provides new benchmarks for high-resolution superpixel segmentation

Abstract

Superpixel segmentation has recently seen important progress benefiting from the advances in differentiable deep learning. However, the very high-resolution superpixel segmentation still remains challenging due to the expensive memory and computation cost, making the current advanced superpixel networks fail to process. In this paper, we devise Patch Calibration Networks (PCNet), aiming to efficiently and accurately implement high-resolution superpixel segmentation. PCNet follows the principle of producing high-resolution output from low-resolution input for saving GPU memory and relieving computation cost. To recall the fine details destroyed by the down-sampling operation, we propose a novel Decoupled Patch Calibration (DPC) branch for collaboratively augment the main superpixel generation branch. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. By sharing the parameters of DPC and main branches, the fine-detailed knowledge learned from high-resolution patches will be transferred to help calibrate the destroyed information. To the best of our knowledge, we make the first attempt to consider the deep-learning-based superpixel generation for high-resolution cases. To facilitate this research, we build evaluation benchmarks from two public datasets and one new constructed one, covering a wide range of diversities from fine-grained human parts to cityscapes. Extensive experiments demonstrate that our PCNet can not only perform favorably against the state-of-the-arts in the quantitative results but also improve the resolution upper bound from 3K to 5K on 1080Ti GPUs.