Solving LLM Repetition Problem in Production: A Comprehensive Study of Multiple Solutions

TL;DR

This paper investigates the repetition problem in Large Language Models during production, analyzing its causes and evaluating multiple practical solutions like beam search, presence penalty, and DPO fine-tuning for real-world deployment.

Contribution

It provides a comprehensive theoretical analysis and experimental validation of various solutions to mitigate LLM repetition issues in production environments.

Findings

Beam Search with early stopping effectively resolves all repetition patterns.

Presence penalty helps specifically with business rule repetition.

DPO fine-tuning offers a universal solution for various repetition cases.

Abstract

The repetition problem, where Large Language Models (LLMs) continuously generate repetitive content without proper termination, poses a critical challenge in production deployments, causing severe performance degradation and system stalling. This paper presents a comprehensive investigation and multiple practical solutions for the repetition problem encountered in real-world batch code interpretation tasks. We identify three distinct repetition patterns: (1) business rule generation repetition, (2) method call relationship analysis repetition, and (3) PlantUML diagram syntax generation repetition. Through rigorous theoretical analysis based on Markov models, we establish that the root cause lies in greedy decoding's inability to escape repetitive loops, exacerbated by self-reinforcement effects. Our comprehensive experimental evaluation demonstrates three viable solutions: (1) Beam Search decoding with early_stopping=True serves as a universal post-hoc mechanism that effectively resolves all three repetition patterns; (2) presence_penalty hyperparameter provides an effective solution specifically for BadCase 1; and (3) Direct Preference Optimization (DPO) fine-tuning offers a universal model-level solution for all three BadCases. The primary value of this work lies in combining first-hand production experience with extensive experimental validation. Our main contributions include systematic theoretical analysis of repetition mechanisms, comprehensive evaluation of multiple solutions with task-specific applicability mapping, identification of early_stopping as the critical parameter for Beam Search effectiveness, and practical production-ready solutions validated in real deployment environments.