Boundary-Aware Segmentation Network for Mobile and Web Applications

TL;DR

This paper introduces BASNet, a boundary-aware segmentation network that achieves highly accurate, real-time image segmentation with sharp boundaries, and demonstrates its effectiveness in practical AR and web applications.

Contribution

The paper presents a novel predict-refine architecture with a hybrid loss for improved boundary accuracy in image segmentation, applicable to mobile and web scenarios.

Findings

Achieves over 70 fps on a single GPU

Performs competitively on salient and camouflaged object segmentation

Enables real-world applications like AR copy-paste and background removal

Abstract

Although deep models have greatly improved the accuracy and robustness of image segmentation, obtaining segmentation results with highly accurate boundaries and fine structures is still a challenging problem. In this paper, we propose a simple yet powerful Boundary-Aware Segmentation Network (BASNet), which comprises a predict-refine architecture and a hybrid loss, for highly accurate image segmentation. The predict-refine architecture consists of a densely supervised encoder-decoder network and a residual refinement module, which are respectively used to predict and refine a segmentation probability map. The hybrid loss is a combination of the binary cross entropy, structural similarity and intersection-over-union losses, which guide the network to learn three-level (ie, pixel-, patch- and map- level) hierarchy representations. We evaluate our BASNet on two reverse tasks including salient object segmentation, camouflaged object segmentation, showing that it achieves very competitive performance with sharp segmentation boundaries. Importantly, BASNet runs at over 70 fps on a single GPU which benefits many potential real applications. Based on BASNet, we further developed two (close to) commercial applications: AR COPY & PASTE, in which BASNet is integrated with augmented reality for "COPYING" and "PASTING" real-world objects, and OBJECT CUT, which is a web-based tool for automatic object background removal. Both applications have already drawn huge amount of attention and have important real-world impacts. The code and two applications will be publicly available at: https://github.com/NathanUA/BASNet.