Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition

TL;DR

This paper introduces a multi-level adaptive ROI and graph learning framework for facial action unit recognition, improving regional feature representation and AU relation modeling to achieve superior performance.

Contribution

The proposed MARGL framework adaptively adjusts AU regions and models intra- and inter-level AU relations using graph convolution, advancing facial AU recognition methods.

Findings

Significantly outperforms previous state-of-the-art methods on BP4D and DISFA datasets.

Effectively models intra-level and inter-level AU relations.

Enhances regional feature representation for better AU recognition.

Abstract

In facial action unit (AU) recognition tasks, regional feature learning and AU relation modeling are two effective aspects which are worth exploring. However, the limited representation capacity of regional features makes it difficult for relation models to embed AU relationship knowledge. In this paper, we propose a novel multi-level adaptive ROI and graph learning (MARGL) framework to tackle this problem. Specifically, an adaptive ROI learning module is designed to automatically adjust the location and size of the predefined AU regions. Meanwhile, besides relationship between AUs, there exists strong relevance between regional features across multiple levels of the backbone network as level-wise features focus on different aspects of representation. In order to incorporate the intra-level AU relation and inter-level AU regional relevance simultaneously, a multi-level AU relation graph is constructed and graph convolution is performed to further enhance AU regional features of each level. Experiments on BP4D and DISFA demonstrate the proposed MARGL significantly outperforms the previous state-of-the-art methods.