Is Your Large Language Model Knowledgeable or a Choices-Only Cheater?

TL;DR

This paper investigates whether large language models rely on shortcuts in multiple-choice question answering and finds that they do not, suggesting leaderboard rankings reflect genuine understanding rather than shortcut exploitation.

Contribution

The paper introduces a graph mining method to create contrast sets from existing datasets, enabling unbiased evaluation of LLMs' reliance on shortcuts in MCQA.

Findings

LLMs do not rely on choices-only shortcuts when given full question context

Contrast sets reveal that high leaderboard scores are not solely due to shortcut exploitation

Graph mining effectively generates unbiased contrast sets from existing datasets

Abstract

Recent work shows that large language models (LLMs) can answer multiple-choice questions using only the choices, but does this mean that MCQA leaderboard rankings of LLMs are largely influenced by abilities in choices-only settings? To answer this, we use a contrast set that probes if LLMs over-rely on choices-only shortcuts in MCQA. While previous works build contrast sets via expensive human annotations or model-generated data which can be biased, we employ graph mining to extract contrast sets from existing MCQA datasets. We use our method on UnifiedQA, a group of six commonsense reasoning datasets with high choices-only accuracy, to build an 820-question contrast set. After validating our contrast set, we test 12 LLMs, finding that these models do not exhibit reliance on choice-only shortcuts when given both the question and choices. Thus, despite the susceptibility~of MCQA to high choices-only accuracy, we argue that LLMs are not obtaining high ranks on MCQA leaderboards just due to their ability to exploit choices-only shortcuts.