Multi-task learning on partially labeled datasets via invariant/equivariant semi-supervised learning

TL;DR

This paper explores invariant and equivariant semi-supervised learning methods, specifically FixMatch and Dense FixMatch, for multi-task models trained on partially labeled datasets, showing improved performance especially with limited labeled data.

Contribution

It demonstrates the effectiveness of invariant and equivariant semi-supervised learning in multi-task settings with partially labeled data, extending existing methods to new scenarios.

Findings

Both methods outperform supervised baselines in most cases.

Significant improvements when fewer labeled samples are available.

Equivariant Dense FixMatch generally yields better results than invariant FixMatch.

Abstract

We investigate the potential of invariant and equivariant semi-supervised learning for addressing the challenges of training multi-task models on partially labeled datasets with differently structured output tasks. Specifically, we use the popular FixMatch method for invariant semi-supervised learning and its equivariant extension Dense FixMatch. We evaluate their performance on the Cityscapes and BDD100K datasets in the context of the prevalent object detection and semantic segmentation tasks in computer vision. We consider varying sizes of the subsets annotated for each task and different overlaps among them. Our results for both invariant and equivariant semi-supervised learning outperform supervised baselines in most situations, with the most significant improvements observed when fewer labeled samples are available for a task and generally better results for the latter approach. Our study suggests that invariant/equivariant learning is a promising general direction for multi-task learning from limited labeled data.