PPJudge: Towards Human-Aligned Assessment of Artistic Painting Process

TL;DR

This paper introduces PPJudge, a Transformer-based model for assessing the artistic painting process, supported by a large-scale dataset, aiming to better align AI evaluation with human judgments of dynamic art creation.

Contribution

The paper presents the first large-scale painting process dataset with expert annotations and a novel Transformer-based model for human-aligned assessment of painting processes.

Findings

Outperforms existing methods in accuracy and robustness

Achieves better alignment with human judgment

Provides new insights into computational creativity

Abstract

Artistic image assessment has become a prominent research area in computer vision. In recent years, the field has witnessed a proliferation of datasets and methods designed to evaluate the aesthetic quality of paintings. However, most existing approaches focus solely on static final images, overlooking the dynamic and multi-stage nature of the artistic painting process. To address this gap, we propose a novel framework for human-aligned assessment of painting processes. Specifically, we introduce the Painting Process Assessment Dataset (PPAD), the first large-scale dataset comprising real and synthetic painting process images, annotated by domain experts across eight detailed attributes. Furthermore, we present PPJudge (Painting Process Judge), a Transformer-based model enhanced with temporally-aware positional encoding and a heterogeneous mixture-of-experts architecture, enabling effective assessment of the painting process. Experimental results demonstrate that our method outperforms existing baselines in accuracy, robustness, and alignment with human judgment, offering new insights into computational creativity and art education.