ScrollTimes: Tracing the Provenance of Paintings as a Window into History

TL;DR

ScrollTimes is a system that combines image processing, language models, and interactive visualization to trace the provenance of Chinese handscroll paintings, providing a comprehensive historical biography for art historians.

Contribution

The paper introduces a novel three-tiered methodology and a visual analysis system, ScrollTimes, for detailed provenance analysis of handscrolls, integrating multiple data sources and expert validation.

Findings

Effective extraction of provenance data from handscrolls

Enhanced interactive analysis for art historians

Validated approach through case studies and expert feedback

Abstract

The study of cultural artifact provenance, tracing ownership and preservation, holds significant importance in archaeology and art history. Modern technology has advanced this field, yet challenges persist, including recognizing evidence from diverse sources, integrating sociocultural context, and enhancing interactive automation for comprehensive provenance analysis. In collaboration with art historians, we examined the handscroll, a traditional Chinese painting form that provides a rich source of historical data and a unique opportunity to explore history through cultural artifacts. We present a three-tiered methodology encompassing artifact, contextual, and provenance levels, designed to create a "Biography" for handscroll. Our approach incorporates the application of image processing techniques and language models to extract, validate, and augment elements within handscroll using various cultural heritage databases. To facilitate efficient analysis of non-contiguous extracted elements, we have developed a distinctive layout. Additionally, we introduce ScrollTimes, a visual analysis system tailored to support the three-tiered analysis of handscroll, allowing art historians to interactively create biographies tailored to their interests. Validated through case studies and expert interviews, our approach offers a window into history, fostering a holistic understanding of handscroll provenance and historical significance.