Promoting Critical Thinking With Domain-Specific Generative AI Provocations

TL;DR

This paper explores how domain-specific generative AI provocations can enhance critical thinking by designing interactive tools that adapt to user expertise and promote meaningful engagement.

Contribution

It introduces two AI-powered tools, ArtBot and Privy, demonstrating how tailored provocations support critical thinking in specific domains.

Findings

Domain-specific provocations foster critical thinking.

Interactive design with user contribution enhances effectiveness.

Adaptive approaches may improve critical thinking support.

Abstract

The evidence on the effects of generative AI (GenAI) on critical thinking is mixed, with studies suggesting both potential harms and benefits depending on its implementation. Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking. Drawing on our experience designing and evaluating two GenAI-powered tools for knowledge work, ArtBot in the domain of fine art interpretation and Privy in the domain of AI privacy, we reflect on how design decisions shape the form and effectiveness of such provocations. Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking. We present participant experiences with both prototypes and discuss how supporting critical thinking may require moving beyond static provocations toward approaches that adapt to user preferences and levels of expertise.