Parametric Image Segmentation of Humans with Structural Shape Priors

TL;DR

This paper introduces a novel parametric max-flow based approach for human image segmentation that incorporates structural shape priors and a large dataset, achieving state-of-the-art results in complex natural scenes.

Contribution

It formulates a sub-modular energy model with class-specific shape constraints and develops a data-driven fusion method for on-the-fly shape prior construction.

Findings

Achieved 20% improvement over top methods on H3D and MPII datasets.

Reduced hypothesis set sizes by up to tenfold.

Demonstrated systematic computation of all model breakpoints in polynomial time.

Abstract

The figure-ground segmentation of humans in images captured in natural environments is an outstanding open problem due to the presence of complex backgrounds, articulation, varying body proportions, partial views and viewpoint changes. In this work we propose class-specific segmentation models that leverage parametric max-flow image segmentation and a large dataset of human shapes. Our contributions are as follows: (1) formulation of a sub-modular energy model that combines class-specific structural constraints and data-driven shape priors, within a parametric max-flow optimization methodology that systematically computes all breakpoints of the model in polynomial time; (2) design of a data-driven class-specific fusion methodology, based on matching against a large training set of exemplar human shapes (100,000 in our experiments), that allows the shape prior to be constructed on-the-fly, for arbitrary viewpoints and partial views. (3) demonstration of state of the art results, in two challenging datasets, H3D and MPII (where figure-ground segmentation annotations have been added by us), where we substantially improve on the first ranked hypothesis estimates of mid-level segmentation methods, by 20%, with hypothesis set sizes that are up to one order of magnitude smaller.