Strategies for Creating Uncertainty in the AI Era to Trigger Students Critical Thinking: Pedagogical Design, Assessment Rubric, and Exam System

TL;DR

This paper proposes pedagogical strategies that leverage AI's limitations to foster critical thinking in students by designing uncertain learning activities, controlling AI behavior during exams, and assessing reasoning processes.

Contribution

It introduces a novel pedagogical framework that integrates controllable AI tools and assessment rubrics to enhance critical thinking through uncertainty-based learning activities.

Findings

AI limitations can be used to create uncertainty in assessments.

Controlling AI behavior during exams prevents shortcut solutions.

The MindMosaicAIExam system supports iterative reasoning and evaluation.

Abstract

Generative AI challenges traditional assessments by allowing students to produce correct answers without demonstrating understanding or reasoning. Rather than prohibiting AI, this work argues that one way to integrate AI into education is by creating uncertain situations with the help of AI models and using thinking-oriented teaching approaches, where uncertainty is a central pedagogical concept for stimulating students critical thinking. Drawing on epistemology and critical thinking research studies, we propose designing learning activities and assessments around the inherent limitations of both AI models and instructors. This encourages students to reason, question, and justify their final answers. We show how explicitly controlling AI behavior during exams (such as preventing direct answers or generating plausible but flawed responses) prevents AI from becoming a shortcut to certainty. To support this pedagogy, we introduce MindMosaicAIExam, an exam system that integrates controllable AI tools and requires students to provide initial answers, critically evaluate AI outputs, and iteratively refine their reasoning. We also present an evaluation rubric designed to assess critical thinking based on students reasoning artifacts collected by the exam system.