CounterRefine: Answer-Conditioned Counterevidence Retrieval for Inference-Time Knowledge Repair in Factual Question Answering

TL;DR

CounterRefine is a lightweight method that improves factual question answering by testing and refining initial answers using answer-conditioned evidence retrieval and validation, leading to more accurate responses.

Contribution

It introduces a novel, minimal repair layer that enhances existing retrieval-augmented generation systems by enabling answer reconsideration and correction through evidence-based refinement.

Findings

Improves correct-answer rate by up to 5.8 points on SimpleQA benchmark.

Changes only 5.6% of outputs in Claude trace, with mostly beneficial corrections.

Demonstrates the importance of answer repair mechanisms in foundation models.

Abstract

In factual question answering, many errors are not failures of access but failures of commitment: the system retrieves relevant evidence, yet still settles on the wrong answer. We present CounterRefine, a lightweight repair layer for short-form RAG that treats the first answer as a hypothesis to test. Given a draft, CounterRefine issues answer-conditioned expansion queries to retrieve candidate-specific evidence, then applies a constrained KEEP or REVISE refinement step whose proposed revisions are accepted only after deterministic validation. The design is intentionally narrow: it adds one evidence-gathering pass and one guarded refinement call rather than replacing the retriever or building a broad agentic system. On the full SimpleQA benchmark, CounterRefine improves a matched one-pass RAG baseline by up to 5.8 correct-rate points; in the full Claude trace, it changes only 5.6% of outputs, with 180 beneficial outcome changes and 8 harmful ones. These findings suggest a simple but important direction for knowledgeable foundation models: beyond accessing evidence, they should also be able to use that evidence to reconsider and, when necessary, repair their own answers.