Strong Baselines for Neural Semi-supervised Learning under Domain Shift

TL;DR

This paper critically reevaluates neural semi-supervised learning methods under domain shift, demonstrating that classic tri-training approaches remain strong baselines and can outperform recent neural methods in certain tasks.

Contribution

The paper introduces a novel multi-task tri-training method that reduces complexity and provides a comprehensive comparison of neural and classic approaches under domain shift.

Findings

Classic tri-training outperforms recent neural methods in some benchmarks.

The proposed multi-task tri-training reduces time and space complexity.

State-of-the-art performance achieved on sentiment analysis.

Abstract

Novel neural models have been proposed in recent years for learning under domain shift. Most models, however, only evaluate on a single task, on proprietary datasets, or compare to weak baselines, which makes comparison of models difficult. In this paper, we re-evaluate classic general-purpose bootstrapping approaches in the context of neural networks under domain shifts vs. recent neural approaches and propose a novel multi-task tri-training method that reduces the time and space complexity of classic tri-training. Extensive experiments on two benchmarks are negative: while our novel method establishes a new state-of-the-art for sentiment analysis, it does not fare consistently the best. More importantly, we arrive at the somewhat surprising conclusion that classic tri-training, with some additions, outperforms the state of the art. We conclude that classic approaches constitute an important and strong baseline.