A Strategy for Label Alignment in Deep Neural Networks

TL;DR

This paper extends label alignment techniques from linear regression to deep neural networks for unsupervised domain adaptation, achieving comparable performance with more stable convergence.

Contribution

It generalizes label alignment for deep learning, providing an alternative adaptation algorithm and demonstrating its effectiveness and stability.

Findings

Achieves comparable performance to mainstream methods

Demonstrates more stable convergence

Provides open-source implementation

Abstract

One recent research demonstrated successful application of the label alignment property for unsupervised domain adaptation in a linear regression settings. Instead of regularizing representation learning to be domain invariant, the research proposed to regularize the linear regression model to align with the top singular vectors of the data matrix from the target domain. In this work we expand upon this idea and generalize it to the case of deep learning, where we derive an alternative formulation of the original adaptation algorithm exploiting label alignment suitable for deep neural network. We also perform experiments to demonstrate that our approach achieves comparable performance to mainstream unsupervised domain adaptation methods while having stabler convergence. All experiments and implementations in our work can be found at the following codebase: https://github.com/xuanrui-work/DeepLabelAlignment.