Human Centred Object Co-Segmentation

TL;DR

This paper introduces a human-centered co-segmentation method that leverages human-object interactions and a fully connected CRF auto-encoder to improve the accuracy of extracting common objects across images.

Contribution

It presents an unsupervised, fully connected CRF auto-encoder incorporating human-object interaction features, a novel approach for co-segmentation that outperforms existing methods.

Findings

More accurate object extraction than state-of-the-art algorithms

Effective use of human-object interaction features

Unsupervised learning approach

Abstract

Co-segmentation is the automatic extraction of the common semantic regions given a set of images. Different from previous approaches mainly based on object visuals, in this paper, we propose a human centred object co-segmentation approach, which uses the human as another strong evidence. In order to discover the rich internal structure of the objects reflecting their human-object interactions and visual similarities, we propose an unsupervised fully connected CRF auto-encoder incorporating the rich object features and a novel human-object interaction representation. We propose an efficient learning and inference algorithm to allow the full connectivity of the CRF with the auto-encoder, that establishes pairwise relations on all pairs of the object proposals in the dataset. Moreover, the auto-encoder learns the parameters from the data itself rather than supervised learning or manually assigned parameters in the conventional CRF. In the extensive experiments on four datasets, we show that our approach is able to extract the common objects more accurately than the state-of-the-art co-segmentation algorithms.