KCIF: Knowledge-Conditioned Instruction Following

TL;DR

This paper investigates how large language models struggle to follow combined knowledge and instruction tasks, revealing significant performance drops especially in smaller models, and introduces a benchmark to evaluate this interaction.

Contribution

The paper introduces a new benchmark dataset and evaluation framework to study the interaction between knowledge and instruction following in LLMs, highlighting their joint challenges.

Findings

Models show a 40-50% performance drop on combined tasks.

Smaller models experience performance drops exceeding 80%.

Large models still struggle significantly with instruction-knowledge interaction.

Abstract

LLM evaluation benchmarks have traditionally separated the testing of knowledge/reasoning capabilities from instruction following. In this work, we study the interaction between knowledge and instruction following, and observe that LLMs struggle to follow simple answer modifying instructions, and are also distracted by instructions that should have no bearing on the original knowledge task answer. We leverage existing multiple-choice answer based knowledge benchmarks and apply a set of simple instructions which include manipulating text (eg.: change case), numeric quantities (eg.: increase value, change formatting), operate on lists (eg.: sort answer candidates) and distractor instructions (eg.: change case of numeric answers). We evaluate models at varying parameter sizes (1B-405B) from different model families and find that, surprisingly, all models report a significant drop in performance on such simple task compositions. While large-sized and frontier models report performance drops of 40-50%, in small and medium sized models the drop is severe (sometimes exceeding 80%). Our results highlight a limitation in the traditional separation of knowledge/reasoning and instruction following, and suggest that joint-study of these capabilities are important. We release our benchmark dataset, evaluation framework code, and results for future work.