Finetuning LLMs for Comparative Assessment Tasks

TL;DR

This paper introduces a finetuning framework for large language models to improve their efficiency and accuracy in comparative assessment tasks in natural language generation, addressing scalability issues of pairwise comparisons.

Contribution

It presents a novel finetuning method that aligns LLM outputs with target comparative probabilities, enhancing performance and efficiency over existing approaches.

Findings

Improved state-of-the-art performance in comparative assessment

Maintains high accuracy with fewer comparisons

Addresses scalability issues of pairwise comparisons

Abstract

Automated assessment in natural language generation is a challenging task. Instruction-tuned large language models (LLMs) have shown promise in reference-free evaluation, particularly through comparative assessment. However, the quadratic computational complexity of pairwise comparisons limits its scalability. To address this, efficient comparative assessment has been explored by applying comparative strategies on zero-shot LLM probabilities. We propose a framework for finetuning LLMs for comparative assessment to align the model's output with the target distribution of comparative probabilities. By training on soft probabilities, our approach improves state-of-the-art performance while maintaining high performance with an efficient subset of comparisons.