Focal-RegionFace: Generating Fine-Grained Multi-attribute Descriptions for Arbitrarily Selected Face Focal Regions

TL;DR

This paper presents Focal-RegionFace, a model that generates detailed multi-attribute descriptions for specific face regions, improving facial analysis by focusing on localized features with a new dataset and a fine-tuned vision-language approach.

Contribution

The paper introduces a new dataset and a fine-tuned vision-language model for region-specific facial attribute analysis, enabling more precise and interpretable facial state recognition.

Findings

Achieves state-of-the-art performance on the new benchmark.

Effectively recognizes multiple facial attributes simultaneously.

Demonstrates improved interpretability and focus on facial regions.

Abstract

In this paper, we introduce an underexplored problem in facial analysis: generating and recognizing multi-attribute natural language descriptions, containing facial action units (AUs), emotional states, and age estimation, for arbitrarily selected face regions (termed FaceFocalDesc). We argue that the system's ability to focus on individual facial areas leads to better understanding and control. To achieve this capability, we construct a new multi-attribute description dataset for arbitrarily selected face regions, providing rich region-level annotations and natural language descriptions. Further, we propose a fine-tuned vision-language model based on Qwen2.5-VL, called Focal-RegionFace for facial state analysis, which incrementally refines its focus on localized facial features through multiple progressively fine-tuning stages, resulting in interpretable age estimation, FAU and emotion detection. Experimental results show that Focal-RegionFace achieves the best performance on the new benchmark in terms of traditional and widely used metrics, as well as new proposed metrics. This fully verifies its effectiveness and versatility in fine-grained multi-attribute face region-focal analysis scenarios.