Thinking Like a Scientist: Can Interactive Simulations Foster Critical AI Literacy?

TL;DR

This study investigates whether interactive simulations can improve AI literacy by engaging learners in scientific inquiry, demonstrating their effectiveness in enhancing understanding of AI concepts like bias and fairness.

Contribution

It provides evidence that interactive, inquiry-based AI tutorials significantly improve learners' understanding and confidence in AI concepts, outperforming traditional methods.

Findings

Interactive simulations increase AI literacy and understanding.

Engagement alone does not predict learning outcomes.

Simulations support better knowledge transfer and confidence.

Abstract

As AI systems shape individual and societal decisions, fostering critical AI literacy is essential. Traditional approaches, such as blog articles, static lessons, and social media discussions, often fail to support deep conceptual understanding and critical engagement. This study examines whether interactive simulations can help learners think like a scientist by engaging them in hypothesis testing, experimentation, and direct observation of AI behavior. In a controlled study with 605 participants, we assess how interactive AI tutorials impact learning of key concepts such as fairness, dataset representativeness, and bias in language models. Results show that interactive simulations effectively enhance AI literacy across topics, supporting greater knowledge transfer and self-reported confidence, though engagement alone does not predict learning. This work contributes to the growing field of AI literacy education, highlighting how interactive, inquiry-driven methodologies can better equip individuals to critically engage with AI in their daily lives.