Training-Free Test-Time Contrastive Learning for Large Language Models

TL;DR

This paper introduces TF-TTCL, a training-free, test-time adaptation framework for large language models that improves reasoning by dynamically distilling and applying semantic rules during inference.

Contribution

It proposes a novel 'Explore-Reflect-Steer' loop enabling frozen LLMs to adapt online without additional training or gradient updates.

Findings

TF-TTCL outperforms zero-shot baselines on reasoning tasks.

The method effectively distills semantic rules from inference trajectories.

Dynamic rule retrieval improves reasoning robustness.

Abstract

Large language models (LLMs) demonstrate strong reasoning capabilities, but their performance often degrades under distribution shift. Existing test-time adaptation (TTA) methods rely on gradient-based updates that require white-box access and need substantial overhead, while training-free alternatives are either static or depend on external guidance. In this paper, we propose Training-Free Test-Time Contrastive Learning TF-TTCL, a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences. Specifically, TF-TTCL implements a dynamic "Explore-Reflect-Steer" loop through three core modules: 1) Semantic Query Augmentation first diversifies problem views via multi-agent role-playing to generate different reasoning trajectories; 2) Contrastive Experience Distillation then captures the semantic gap between superior and inferior trajectories, distilling them into explicit textual rules; and 3) Contextual Rule Retrieval finally activates these stored rules during inference to dynamically steer the frozen LLM toward robust reasoning patterns while avoiding observed errors. Extensive experiments on closed-ended reasoning tasks and open-ended evaluation tasks demonstrate that TF-TTCL consistently outperforms strong zero-shot baselines and representative TTA methods under online evaluation. Code is available at https://github.com/KevinSCUTer/TF-TTCL.