Can you tell a face from a HEVC bitstream?

TL;DR

This paper demonstrates that face detection can be performed directly on HEVC bitstream data using a CNN, eliminating the need for full image reconstruction and saving computational resources.

Contribution

It introduces a novel approach to face detection directly from HEVC bitstream data, bypassing full image decoding.

Findings

Face detection accuracy comparable to traditional methods

CNN trained on HEVC entropy decoder output is effective

Reduces computational resources needed for face detection

Abstract

Image and video analytics are being increasingly used on a massive scale. Not only is the amount of data growing, but the complexity of the data processing pipelines is also increasing, thereby exacerbating the problem. It is becoming increasingly important to save computational resources wherever possible. We focus on one of the poster problems of visual analytics -- face detection -- and approach the issue of reducing the computation by asking: Is it possible to detect a face without full image reconstruction from the High Efficiency Video Coding (HEVC) bitstream? We demonstrate that this is indeed possible, with accuracy comparable to conventional face detection, by training a Convolutional Neural Network on the output of the HEVC entropy decoder.