CERET: Cost-Effective Extrinsic Refinement for Text Generation

TL;DR

CERET is a cost-effective method for refining text generated by large language models, improving quality and accuracy by leveraging semantic stability and uncertainty measures, with lower computational costs than existing self-improvement techniques.

Contribution

Introduces CERET, a novel refinement approach that enhances LLM output quality efficiently using semantic and uncertainty metrics, outperforming existing methods in accuracy and cost-effectiveness.

Findings

CERET improves Rouge-1 by ~1.6% in summarization.

CERET increases question answering hit rate by ~3.5%.

CERET uses only 9.4% of the latency of LLM Self-rerank.

Abstract

Large Language Models (LLMs) are powerful models for generation tasks, but they may not generate good quality outputs in their first attempt. Apart from model fine-tuning, existing approaches to improve prediction accuracy and quality typically involve LLM self-improvement / self-reflection that incorporate feedback from models themselves. Despite their effectiveness, these methods are hindered by their high computational cost and lack of scalability. In this work, we propose CERET, a method for refining text generations by considering semantic stability, entailment and inter-sample uncertainty measures. Experimental results show that CERET outperforms Self-consistency and Self-rerank baselines consistently under various task setups, by ~1.6% in Rouge-1 for abstractive summarization and ~3.5% in hit rate for question answering. Compared to LLM Self-rerank method, our approach only requires 9.4% of its latency and is more cost-effective.