A Survey of Early Exit Deep Neural Networks in NLP

TL;DR

This survey reviews early exit strategies in deep neural networks for NLP, highlighting their ability to reduce inference time and enhance robustness by adaptively classifying samples at different network layers.

Contribution

It provides a comprehensive overview of early exit methods in NLP, detailing their mechanisms, applications, and benefits in resource-constrained and adversarial settings.

Findings

Early exit methods significantly reduce inference latency.

They improve model robustness against adversarial attacks.

Adaptive inference effectively handles mixed difficulty datasets.

Abstract

Deep Neural Networks (DNNs) have grown increasingly large in size to achieve state of the art performance across a wide range of tasks. However, their high computational requirements make them less suitable for resource-constrained applications. Also, real-world datasets often consist of a mixture of easy and complex samples, necessitating adaptive inference mechanisms that account for sample difficulty. Early exit strategies offer a promising solution by enabling adaptive inference, where simpler samples are classified using the initial layers of the DNN, thereby accelerating the overall inference process. By attaching classifiers at different layers, early exit methods not only reduce inference latency but also improve the model robustness against adversarial attacks. This paper presents a comprehensive survey of early exit methods and their applications in NLP.