Teaching People LLM's Errors and Getting it Right
Nathan Stringham, Fateme Hashemi Chaleshtori, Xinyuan Yan, Zhichao Xu, Bei Wang, Ana Marasovi\'c

TL;DR
This paper investigates why teaching users LLM failure patterns has limited success, analyzing failure detection methods and proposing a new metric to better evaluate teaching effectiveness, ultimately showing potential for reducing overreliance.
Contribution
The paper provides an in-depth analysis of failure pattern teaching in LLMs, introduces criteria for identifying failure groups, and proposes a new metric for assessing teaching effectiveness.
Findings
Failure patterns exist but are hard to surface automatically.
Prompting and embedding methods show mixed success in identifying failures.
A new metric improves assessment of teaching effectiveness.
Abstract
People use large language models (LLMs) when they should not. This is partly because they see LLMs compose poems and answer intricate questions, so they understandably, but incorrectly, assume LLMs won't stumble on basic tasks like simple arithmetic. Prior work has tried to address this by clustering instance embeddings into regions where an LLM is likely to fail and automatically describing patterns in these regions. The found failure patterns are taught to users to mitigate their overreliance. Yet, this approach has not fully succeeded. In this analysis paper, we aim to understand why. We first examine whether the negative result stems from the absence of failure patterns. We group instances in two datasets by their meta-labels and evaluate an LLM's predictions on these groups. We then define criteria to flag groups that are sizable and where the LLM is error-prone, and find…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Artificial Intelligence in Healthcare and Education · Ethics and Social Impacts of AI
