Hair Segmentation on Time-of-Flight RGBD Images

TL;DR

This paper introduces a novel method for hair segmentation using ToF RGBD images, leveraging unique noise patterns caused by scattering and reflection, and demonstrates superior accuracy over RGB-only methods.

Contribution

The work presents the first ToF RGBD hair dataset and a deep learning approach that combines ToF noise analysis with RGB data for improved hair segmentation.

Findings

Outperforms RGB-based methods in accuracy and robustness

Effectively handles dark hair and challenging background scenarios

Provides a new dataset of 20k+ ToF RGBD hair images

Abstract

Robust segmentation of hair from portrait images remains challenging: hair does not conform to a uniform shape, style or even color; dark hair in particular lacks features. We present a novel computational imaging solution that tackles the problem from both input and processing fronts. We explore using Time-of-Flight (ToF) RGBD sensors on recent mobile devices. We first conduct a comprehensive analysis to show that scattering and inter-reflection cause different noise patterns on hair vs. non-hair regions on ToF images, by changing the light path and/or combining multiple paths. We then develop a deep network based approach that employs both ToF depth map and the RGB gradient maps to produce an initial hair segmentation with labeled hair components. We then refine the result by imposing ToF noise prior under the conditional random field. We collect the first ToF RGBD hair dataset with 20k+ head images captured on 30 human subjects with a variety of hairstyles at different view angles. Comprehensive experiments show that our approach outperforms the RGB based techniques in accuracy and robustness and can handle traditionally challenging cases such as dark hair, similar hair/background, similar hair/foreground, etc.