Understanding Compositional Structures in Art Historical Images using Pose and Gaze Priors

TL;DR

This paper presents a computer vision approach inspired by art historical theories to automatically analyze compositional structures in artworks, focusing on action regions and pose-based segmentation without training, aiding art analysis and machine understanding.

Contribution

The work introduces a novel, training-free method for detecting compositional elements in artworks using pose and gaze priors, inspired by Max Imdahl's theories.

Findings

High correlation with expert assessments

Demonstrates domain-agnostic applicability

Open-sourced implementation available

Abstract

Image compositions as a tool for analysis of artworks is of extreme significance for art historians. These compositions are useful in analyzing the interactions in an image to study artists and their artworks. Max Imdahl in his work called Ikonik, along with other prominent art historians of the 20th century, underlined the aesthetic and semantic importance of the structural composition of an image. Understanding underlying compositional structures within images is challenging and a time consuming task. Generating these structures automatically using computer vision techniques (1) can help art historians towards their sophisticated analysis by saving lot of time; providing an overview and access to huge image repositories and (2) also provide an important step towards an understanding of man made imagery by machines. In this work, we attempt to automate this process using the existing state of the art machine learning techniques, without involving any form of training. Our approach, inspired by Max Imdahl's pioneering work, focuses on two central themes of image composition: (a) detection of action regions and action lines of the artwork; and (b) pose-based segmentation of foreground and background. Currently, our approach works for artworks comprising of protagonists (persons) in an image. In order to validate our approach qualitatively and quantitatively, we conduct a user study involving experts and non-experts. The outcome of the study highly correlates with our approach and also demonstrates its domain-agnostic capability. We have open-sourced the code at https://github.com/image-compostion-canvas-group/image-compostion-canvas.