Learning Improved Representations by Transferring Incomplete Evidence Across Heterogeneous Tasks

TL;DR

This paper evaluates Evidence Transfer, a representation learning method that leverages external categorical evidence, demonstrating its robustness and effectiveness in scenarios with incomplete or noisy evidence across various tasks.

Contribution

The paper introduces and empirically assesses Evidence Transfer, a novel method for improving representations using incomplete external evidence, showing robustness against noise and evidence incompleteness.

Findings

Evidence transfer improves representation quality with incomplete evidence.

The method remains robust under various levels of evidence noise.

Performance gains are consistent across different task types.

Abstract

Acquiring ground truth labels for unlabelled data can be a costly procedure, since it often requires manual labour that is error-prone. Consequently, the available amount of labelled data is increasingly reduced due to the limitations of manual data labelling. It is possible to increase the amount of labelled data samples by performing automated labelling or crowd-sourcing the annotation procedure. However, they often introduce noise or uncertainty in the labelset, that leads to decreased performance of supervised deep learning methods. On the other hand, weak supervision methods remain robust during noisy labelsets or can be effective even with low amounts of labelled data. In this paper we evaluate the effectiveness of a representation learning method that uses external categorical evidence called "Evidence Transfer", against low amount of corresponding evidence termed as incomplete evidence. Evidence transfer is a robust solution against external unknown categorical evidence that can introduce noise or uncertainty. In our experimental evaluation, evidence transfer proves to be effective and robust against different levels of incompleteness, for two types of incomplete evidence.