Alignment Adapter to Improve the Performance of Compressed Deep Learning Models

TL;DR

This paper introduces Alignment Adapter (AlAd), a lightweight module that enhances the performance of compressed deep learning models by aligning their embeddings with those of larger models, applicable across various compression methods.

Contribution

The paper presents AlAd, a novel, flexible, and compression-agnostic adapter that improves compressed model performance without significant overhead.

Findings

AlAd significantly boosts compressed model accuracy.

AlAd maintains low size and latency overhead.

Effective across multiple NLP tasks.

Abstract

Compressed Deep Learning (DL) models are essential for deployment in resource-constrained environments. But their performance often lags behind their large-scale counterparts. To bridge this gap, we propose Alignment Adapter (AlAd): a lightweight, sliding-window-based adapter. It aligns the token-level embeddings of a compressed model with those of the original large model. AlAd preserves local contextual semantics, enables flexible alignment across differing dimensionalities or architectures, and is entirely agnostic to the underlying compression method. AlAd can be deployed in two ways: as a plug-and-play module over a frozen compressed model, or by jointly fine-tuning AlAd with the compressed model for further performance gains. Through experiments on BERT-family models across three token-level NLP tasks, we demonstrate that AlAd significantly boosts the performance of compressed models with only marginal overhead in size and latency.