Fast Deep Matting for Portrait Animation on Mobile Phone

TL;DR

This paper introduces a fast, real-time deep learning method for portrait image matting on mobile phones, enabling automatic, high-quality alpha matte generation without user interaction at 15 fps.

Contribution

The authors develop a lightweight convolutional network with a feathering block for real-time, automatic portrait matting on mobile devices, outperforming traditional methods in speed and usability.

Findings

Achieves real-time portrait matting at 15 fps on mobile devices.

Produces alpha mattes comparable to state-of-the-art methods.

Operates automatically without user interaction.

Abstract

Image matting plays an important role in image and video editing. However, the formulation of image matting is inherently ill-posed. Traditional methods usually employ interaction to deal with the image matting problem with trimaps and strokes, and cannot run on the mobile phone in real-time. In this paper, we propose a real-time automatic deep matting approach for mobile devices. By leveraging the densely connected blocks and the dilated convolution, a light full convolutional network is designed to predict a coarse binary mask for portrait images. And a feathering block, which is edge-preserving and matting adaptive, is further developed to learn the guided filter and transform the binary mask into alpha matte. Finally, an automatic portrait animation system based on fast deep matting is built on mobile devices, which does not need any interaction and can realize real-time matting with 15 fps. The experiments show that the proposed approach achieves comparable results with the state-of-the-art matting solvers.