Life Cycle-Aware Evaluation of Knowledge Distillation for Machine Translation: Environmental Impact and Translation Quality Trade-offs

TL;DR

This paper evaluates knowledge distillation in machine translation by analyzing both translation quality and environmental impact, providing a comprehensive protocol for selecting methods based on compute and quality trade-offs.

Contribution

It introduces a life cycle-aware assessment framework that considers environmental impact alongside translation quality for KD methods in machine translation.

Findings

Distillation overhead is dominant at small deployment volumes.

Inference costs dominate at large scales, affecting KD benefits.

Word-level distillation offers better footprint-quality trade-offs than sequence-level.

Abstract

Knowledge distillation (KD) is a tool to compress a larger system (teacher) into a smaller one (student). In machine translation, studies typically report only the translation quality of the student and omit the computational complexity of performing KD, making it difficult to select among the many available KD choices under compute-induced constraints. In this study, we evaluate representative KD methods by considering both translation quality and computational cost. We express computational cost as a carbon footprint using the machine learning life cycle assessment (MLCA) tool. This assessment accounts for runtime operational emissions and amortized hardware production costs throughout the KD model life cycle (teacher training, distillation, and inference). We find that (i) distillation overhead dominates the total footprint at small deployment volumes, (ii) inference dominates at scale, making KD beneficial only beyond a task-dependent usage threshold, and (iii) word-level distillation typically offers more favorable footprint-quality trade-offs than sequence-level distillation. Our protocol provides reproducible guidance for selecting KD methods under explicit quality and compute-induced constraints.