A Holistic Approach for Data-Driven Object Cutout

TL;DR

This paper introduces a holistic, deep learning-based method for automatic object cutout in images, leveraging global shape priors and class-specific neural networks to improve accuracy over existing low-level analysis techniques.

Contribution

The paper presents a novel end-to-end pipeline that uses a deep neural network trained on specific object classes to produce probability maps for accurate object segmentation.

Findings

Significantly outperforms state-of-the-art methods on benchmark datasets.

Effectively leverages global shape priors for improved cutout accuracy.

Provides an automatic, class-specific object segmentation approach.

Abstract

Object cutout is a fundamental operation for image editing and manipulation, yet it is extremely challenging to automate it in real-world images, which typically contain considerable background clutter. In contrast to existing cutout methods, which are based mainly on low-level image analysis, we propose a more holistic approach, which considers the entire shape of the object of interest by leveraging higher-level image analysis and learnt global shape priors. Specifically, we leverage a deep neural network (DNN) trained for objects of a particular class (chairs) for realizing this mechanism. Given a rectangular image region, the DNN outputs a probability map (P-map) that indicates for each pixel inside the rectangle how likely it is to be contained inside an object from the class of interest. We show that the resulting P-maps may be used to evaluate how likely a rectangle proposal is to contain an instance of the class, and further process good proposals to produce an accurate object cutout mask. This amounts to an automatic end-to-end pipeline for catergory-specific object cutout. We evaluate our approach on segmentation benchmark datasets, and show that it significantly outperforms the state-of-the-art on them.