# ArtFace: Towards Historical Portrait Face Identification via Model Adaptation

## TL;DR

This paper explores how fine-tuning foundation models and combining their embeddings with traditional facial recognition methods can significantly improve the identification of sitters in historical paintings, addressing challenges like style variation and domain shift.

## Contribution

It introduces a novel approach that integrates foundation model embeddings with conventional facial recognition networks to enhance artwork face identification.

## Key findings

- Foundation models improve recognition accuracy in paintings.
- Combining embeddings outperforms traditional methods.
- Significant gains over state-of-the-art techniques.

## Abstract

Identifying sitters in historical paintings is a key task for art historians, offering insight into their lives and how they chose to be seen. However, the process is often subjective and limited by the lack of data and stylistic variations. Automated facial recognition is capable of handling challenging conditions and can assist, but while traditional facial recognition models perform well on photographs, they struggle with paintings due to domain shift and high intra-class variation. Artistic factors such as style, skill, intent, and influence from other works further complicate recognition. In this work, we investigate the potential of foundation models to improve facial recognition in artworks. By fine-tuning foundation models and integrating their embeddings with those from conventional facial recognition networks, we demonstrate notable improvements over current state-of-the-art methods. Our results show that foundation models can bridge the gap where traditional methods are ineffective. Paper page at https://www.idiap.ch/paper/artface/