Critical-Reflective Human-AI Collaboration: Exploring Computational Tools for Art Historical Image Retrieval

TL;DR

This paper investigates how to design computational tools that actively support critical reflection, a key aspect of meaningful human-AI collaboration in art historical research, through qualitative studies with art historians using computer vision tools.

Contribution

It introduces a framework for integrating critical reflection into human-AI collaboration and provides four design implications based on empirical research with art historians.

Findings

Critical reflection is essential for meaningful collaboration.

Current tools do not fully support critical reflection.

Four design implications for enhancing critical reflection in human-AI tools.

Abstract

Just as other disciplines, the humanities explore how computational research approaches and tools can meaningfully contribute to scholarly knowledge production. We approach the design of computational tools through the analytical lens of 'human-AI collaboration.' However, there is no generalizable concept of what constitutes 'meaningful' human-AI collaboration. In terms of genuinely human competencies, we consider criticality and reflection as guiding principles of scholarly knowledge production. Although (designing for) reflection is a recurring topic in CSCW and HCI discourses, it has not been centered in work on human-AI collaboration. We posit that integrating both concepts is a viable approach to supporting 'meaningful' human-AI collaboration in the humanities. Our research, thus, is guided by the question of how critical reflection can be enabled in human-AI collaboration. We address this question with a use case that centers on computer vision (CV) tools for art historical image retrieval. Specifically, we conducted a qualitative interview study with art historians and extended the interviews with a think-aloud software exploration. We observed and recorded our participants' interaction with a ready-to-use CV tool in a possible research scenario. We found that critical reflection, indeed, constitutes a core prerequisite for 'meaningful' human-AI collaboration in humanities research contexts. However, we observed that critical reflection was not fully realized during interaction with the CV tool. We interpret this divergence as supporting our hypothesis that computational tools need to be intentionally designed in such a way that they actively scaffold and support critical reflection during interaction. Based on our findings, we suggest four empirically grounded design implications for 'critical-reflective human-AI collaboration'.