Co-training for Low Resource Scientific Natural Language Inference

TL;DR

This paper introduces a co-training approach that leverages training dynamics to weight automatically labeled data, improving scientific natural language inference performance in low-resource settings.

Contribution

It proposes a novel co-training method that uses classifier behavior over epochs to assign importance weights to noisy labels, enhancing semi-supervised learning for scientific NLI.

Findings

Achieved 1.5% macro F1 improvement over baseline

Outperformed several strong SSL baselines

Effectively utilizes noisy automatically labeled data

Abstract

Scientific Natural Language Inference (NLI) is the task of predicting the semantic relation between a pair of sentences extracted from research articles. The automatic annotation method based on distant supervision for the training set of SciNLI (Sadat and Caragea, 2022b), the first and most popular dataset for this task, results in label noise which inevitably degenerates the performance of classifiers. In this paper, we propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels, reflective of the manner they are used in the subsequent training epochs. That is, unlike the existing semi-supervised learning (SSL) approaches, we consider the historical behavior of the classifiers to evaluate the quality of the automatically annotated labels. Furthermore, by assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data, while ensuring that the noisy labels have a minimal impact on model training. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines. We make our code and data available on Github.