Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding

TL;DR

Thunder-KoNUBench is a new Korean negation benchmark that evaluates and improves large language models' understanding of negation, addressing a key challenge in Korean NLP.

Contribution

The paper introduces Thunder-KoNUBench, the first corpus-aligned benchmark for Korean negation, and demonstrates how fine-tuning on it enhances LLMs' negation comprehension.

Findings

LLMs' performance degrades under negation in Korean.

Fine-tuning on Thunder-KoNUBench improves negation understanding.

Model size and instruction tuning influence negation performance.

Abstract

Although negation is known to challenge large language models (LLMs), benchmarks for evaluating negation understanding, especially in Korean, are scarce. We conduct a corpus-based analysis of Korean negation and show that LLM performance degrades under negation. We then introduce Thunder-KoNUBench, a sentence-level benchmark that reflects the empirical distribution of Korean negation phenomena. Evaluating 47 LLMs, we analyze the effects of model size and instruction tuning, and show that fine-tuning on Thunder-KoNUBench improves negation understanding and broader contextual comprehension in Korean.