# Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging

## TL;DR

This paper introduces a method that combines pre-trained and randomly initialized units to improve domain adaptation in POS tagging, especially for social media texts, achieving state-of-the-art results.

## Contribution

It proposes augmenting target networks with random units alongside pre-trained ones to enhance adaptation to low-resource domains.

## Key findings

- Achieves state-of-the-art POS tagging performance on social media datasets.
- Enhances domain adaptation by combining pre-trained and random units.
- Demonstrates effectiveness on three benchmark datasets.

## Abstract

Fine-tuning neural networks is widely used to transfer valuable knowledge from high-resource to low-resource domains. In a standard fine-tuning scheme, source and target problems are trained using the same architecture. Although capable of adapting to new domains, pre-trained units struggle with learning uncommon target-specific patterns. In this paper, we propose to augment the target-network with normalised, weighted and randomly initialised units that beget a better adaptation while maintaining the valuable source knowledge. Our experiments on POS tagging of social media texts (Tweets domain) demonstrate that our method achieves state-of-the-art performances on 3 commonly used datasets.