Facial Expression Recognition System Using DNN Accelerator with Multi-threading on FPGA

TL;DR

This paper presents a stand-alone facial expression recognition system on FPGA that uses a multi-threaded DNN accelerator to improve accuracy and throughput, replacing CPU-based face detection with a DPU-based approach.

Contribution

The work introduces a multi-threaded DPU-based facial detection and recognition system on FPGA, enhancing efficiency and accuracy over previous CPU and dedicated circuit methods.

Findings

Achieved 25 FPS system throughput.

Increased throughput per power consumption by 2.4 times.

Enabled efficient resource utilization on FPGA.

Abstract

In this paper, we implement a stand-alone facial expression recognition system on an SoC FPGA with multi-threading using a Deep learning Processor Unit (DPU). The system consists of two steps: one for face detection step and one for facial expression recognition. In the previous work, the Haar Cascade detector was run on a CPU in the face detection step due to FPGA resource limitations, but this detector is less accurate for profile and variable illumination condition images. Moreover, the previous work used a dedicated circuit accelerator, so running a second DNN inference for face detection on the FPGA would require the addition of a new accelerator. As an alternative to this approach, we run the two inferences by DNN on a DPU, which is a general-purpose CNN accelerator of the systolic array type. Our method for face detection using DenseBox and facial expression recognition using CNN on the same DPU enables the efficient use of FPGA resources while maintaining a small circuit size. We also developed a multi-threading technique that improves the overall throughput while increasing the DPU utilization efficiency. With this approach, we achieved an overall system throughput of 25 FPS and a throughput per power consumption of 2.4 times.