Learning to Segment Human Body Parts with Synthetically Trained Deep Convolutional Networks

TL;DR

This paper introduces a synthetic data-driven deep learning framework for human body part segmentation that achieves state-of-the-art results without relying on real annotated data, utilizing a novel data generation and pre-processing pipeline.

Contribution

The paper presents a synthetic data generation pipeline using a game engine and a new pre-processing module to improve human body part segmentation with deep networks trained solely on synthetic data.

Findings

Outperforms existing segmentation tools on real data

Robust to illumination changes due to pre-processing

Synthetic training data suffices for high-quality segmentation

Abstract

This paper presents a new framework for human body part segmentation based on Deep Convolutional Neural Networks trained using only synthetic data. The proposed approach achieves cutting-edge results without the need of training the models with real annotated data of human body parts. Our contributions include a data generation pipeline, that exploits a game engine for the creation of the synthetic data used for training the network, and a novel pre-processing module, that combines edge response maps and adaptive histogram equalization to guide the network to learn the shape of the human body parts ensuring robustness to changes in the illumination conditions. For selecting the best candidate architecture, we perform exhaustive tests on manually annotated images of real human body limbs. We further compare our method against several high-end commercial segmentation tools on the body parts segmentation task. The results show that our method outperforms the other models by a significant margin. Finally, we present an ablation study to validate our pre-processing module. With this paper, we release an implementation of the proposed approach along with the acquired datasets.