Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks

TL;DR

This paper assesses the robustness of LLM-based educational tutors against adversarial student attacks, proposing a benchmark and defense strategies to improve answer leakage resistance.

Contribution

It introduces a fine-tuned adversarial student agent for benchmarking tutor robustness and evaluates simple defenses to mitigate answer leakage.

Findings

Adversarial student agents often fail to effectively jailbreak tutors.

Fine-tuned adversarial agents can reliably test tutor robustness.

Simple defense strategies can significantly reduce answer leakage.

Abstract

Large Language Models (LLMs) are increasingly used in education, yet their default helpfulness often conflicts with pedagogical principles. Prior work evaluates pedagogical quality via answer leakage-the disclosure of complete solutions instead of scaffolding-but typically assumes well-intentioned learners, leaving tutor robustness under student misuse largely unexplored. In this paper, we study scenarios where students behave adversarially and aim to obtain the correct answer from the tutor. We evaluate a broad set of LLM-based tutor models, including different model families, pedagogically aligned models, and a multi-agent design, under a range of adversarial student attacks. We adapt six groups of adversarial and persuasive techniques to the educational setting and use them to probe how likely a tutor is to reveal the final answer. We evaluate answer leakage robustness using different types of in-context adversarial student agents, finding that they often fail to carry out effective attacks. We therefore introduce an adversarial student agent that we fine-tune to jailbreak LLM-based tutors, which we propose as the core of a standardized benchmark for evaluating tutor robustness. Finally, we present simple but effective defense strategies that reduce answer leakage and strengthen the robustness of LLM-based tutors in adversarial scenarios.