Dual-Label Learning With Irregularly Present Labels

TL;DR

This paper introduces Dual-Label Learning (DLL), a novel framework for multi-task learning with irregularly missing labels, utilizing a dual-tower architecture to improve label prediction accuracy and robustness.

Contribution

The paper proposes a new dual-function system and dual-tower model architecture for multi-task learning with irregular labels, explicitly modeling label correlation and imputing missing labels during training.

Findings

DLL outperforms baseline methods with up to 9.6% F1-score improvement.

Maintains robust performance with up to 60% missing labels.

Achieves better results than baselines at lower missing rates (10%).

Abstract

In multi-task learning, labels are often missing irregularly across samples, which can be fully labeled, partially labeled or unlabeled. The irregular label presence often appears in scientific studies due to experimental limitations. It triggers a demand for a new training and inference mechanism that could accommodate irregularly present labels and maximize their utility. This work focuses on the two-label learning task and proposes a novel training and inference framework, Dual-Label Learning (DLL). The DLL framework formulates the problem into a dual-function system, in which the two functions should simultaneously satisfy standard supervision, structural duality and probabilistic duality. DLL features a dual-tower model architecture that allows for explicit information exchange between labels, aimed at maximizing the utility of partially available labels. During training, missing labels are imputed as part of the forward propagation process, while during inference, labels are predicted jointly as unknowns of a bivariate system of equations. Our theoretical analysis guarantees the feasibility of DLL, and extensive experiments are conducted to verify that by explicitly modeling label correlation and maximizing label utility, our method makes consistently better prediction than baseline approaches by up to 9.6% gain in F1-score or 10.2% reduction in MAPE. Remarkably, DLL maintains robust performance at a label missing rate of up to 60%, achieving even better results than baseline approaches at lower missing rates down to only 10%.