CognArtive: Large Language Models for Automating Art Analysis and Decoding Aesthetic Elements

TL;DR

This paper explores how Large Language Models can automate detailed art analysis, decoding aesthetic elements and patterns across artworks to reveal historical and stylistic evolutions.

Contribution

It introduces a framework leveraging LLMs for high-throughput, detailed formal art analysis and visualizes the evolving patterns in artworks over time.

Findings

LLMs can interpret artistic expressions and visual elements.

Patterns in art evolve systematically across periods.

Interactive visualizations enhance understanding of art analysis.

Abstract

Art, as a universal language, can be interpreted in diverse ways, with artworks embodying profound meanings and nuances. The advent of Large Language Models (LLMs) and the availability of Multimodal Large Language Models (MLLMs) raise the question of how these transformative models can be used to assess and interpret the artistic elements of artworks. While research has been conducted in this domain, to the best of our knowledge, a deep and detailed understanding of the technical and expressive features of artworks using LLMs has not been explored. In this study, we investigate the automation of a formal art analysis framework to analyze a high-throughput number of artworks rapidly and examine how their patterns evolve over time. We explore how LLMs can decode artistic expressions, visual elements, composition, and techniques, revealing emerging patterns that develop across periods. Finally, we discuss the strengths and limitations of LLMs in this context, emphasizing their ability to process vast quantities of art-related data and generate insightful interpretations. Due to the exhaustive and granular nature of the results, we have developed interactive data visualizations, available online https://cognartive.github.io/, to enhance understanding and accessibility.