Artistic Instance-Aware Image Filtering by Convolutional Neural Networks

TL;DR

This paper introduces an instance-aware image filtering method using neural networks, enabling selective artistic effects on foreground objects or backgrounds with flexibility and fast processing.

Contribution

It leverages Mask R-CNN for instance segmentation to apply artistic effects selectively, offering a novel, flexible, and efficient approach compared to traditional whole-image methods.

Findings

Effective separation of foreground and background for artistic filtering

Supports multiple artistic effects with easy addition of new filters

Achieves fast processing with a simple user interface

Abstract

In the recent years, public use of artistic effects for editing and beautifying images has encouraged researchers to look for new approaches to this task. Most of the existing methods apply artistic effects to the whole image. Exploitation of neural network vision technologies like object detection and semantic segmentation could be a new viewpoint in this area. In this paper, we utilize an instance segmentation neural network to obtain a class mask for separately filtering the background and foreground of an image. We implement a top prior-mask selection to let us select an object class for filtering purpose. Different artistic effects are used in the filtering process to meet the requirements of a vast variety of users. Also, our method is flexible enough to allow the addition of new filters. We use pre-trained Mask R-CNN instance segmentation on the COCO dataset as the segmentation network. Experimental results on the use of different filters are performed. System's output results show that this novel approach can create satisfying artistic images with fast operation and simple interface.