Unidirectional Thin Adapter for Efficient Adaptation of Deep Neural Networks

TL;DR

This paper introduces UDTA, a lightweight adapter that enables efficient domain adaptation of deep neural networks without altering the backbone, reducing computation and training time while maintaining or improving accuracy.

Contribution

The paper presents a novel unidirectional thin adapter (UDTA) that adapts pre-trained networks efficiently without backpropagating through the backbone, allowing multi-task adaptation with minimal computation.

Findings

UDTA reduces training computation significantly.

UDTA achieves comparable or better accuracy than traditional adapters.

UDTA enables a single backbone to adapt to multiple tasks independently.

Abstract

In this paper, we propose a new adapter network for adapting a pre-trained deep neural network to a target domain with minimal computation. The proposed model, unidirectional thin adapter (UDTA), helps the classifier adapt to new data by providing auxiliary features that complement the backbone network. UDTA takes outputs from multiple layers of the backbone as input features but does not transmit any feature to the backbone. As a result, UDTA can learn without computing the gradient of the backbone, which saves computation for training significantly. In addition, since UDTA learns the target task without modifying the backbone, a single backbone can adapt to multiple tasks by learning only UDTAs separately. In experiments on five fine-grained classification datasets consisting of a small number of samples, UDTA significantly reduced computation and training time required for backpropagation while showing comparable or even improved accuracy compared with conventional adapter models.