Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation

TL;DR

This paper introduces RaDis, a novel rationale distillation method that improves LLM translation skills without sacrificing their general abilities, by using generated rationales to prevent forgetting during fine-tuning.

Contribution

RaDis leverages self-generated rationales to enhance translation performance while preserving LLMs' broad capabilities, addressing limitations of traditional fine-tuning methods.

Findings

Improved translation accuracy demonstrated in experiments.

Maintained general abilities across multiple NLP tasks.

Reduced catastrophic forgetting during fine-tuning.

Abstract

Large Language Models (LLMs) have achieved impressive results across numerous NLP tasks but still encounter difficulties in machine translation. Traditional methods to improve translation have typically involved fine-tuning LLMs using parallel corpora. However, vanilla fine-tuning often leads to catastrophic forgetting of the instruction-following capabilities and alignment with human preferences, compromising their broad general abilities and introducing potential security risks. These abilities, which are developed using proprietary and unavailable training data, make existing continual instruction tuning methods ineffective. To overcome this issue, we propose a novel approach called RaDis (Rationale Distillation). RaDis harnesses the strong generative capabilities of LLMs to create rationales for training data, which are then "replayed" to prevent forgetting. These rationales encapsulate general knowledge and safety principles, acting as self-distillation targets to regulate the training process. By jointly training on both reference translations and self-generated rationales, the model can learn new translation skills while preserving its overall general abilities. Extensive experiments demonstrate that our method enhances machine translation performance while maintaining the broader capabilities of LLMs across other tasks. This work presents a pathway for creating more versatile LLMs that excel in specialized tasks without compromising generality and safety.