NexusFlow: Unifying Disparate Tasks under Partial Supervision via Invertible Flow Networks

TL;DR

NexusFlow is a versatile framework that unifies diverse tasks under partial supervision by aligning feature distributions with invertible coupling layers, enabling effective knowledge transfer across structurally different tasks.

Contribution

It introduces a novel invertible coupling layer approach for partial multi-task learning, effectively handling structurally diverse tasks without loss of expressive capacity.

Findings

Sets new state-of-the-art on nuScenes for domain-partitioned autonomous driving tasks.

Achieves consistent improvements on NYUv2 across segmentation, depth, and surface normals.

Demonstrates broad applicability to diverse multi-task learning scenarios.

Abstract

Partially Supervised Multi-Task Learning (PS-MTL) aims to leverage knowledge across tasks when annotations are incomplete. Existing approaches, however, have largely focused on the simpler setting of homogeneous, dense prediction tasks, leaving the more realistic challenge of learning from structurally diverse tasks unexplored. To this end, we introduce NexusFlow, a novel, lightweight, and plug-and-play framework effective in both settings. NexusFlow introduces a set of surrogate networks with invertible coupling layers to align the latent feature distributions of tasks, creating a unified representation that enables effective knowledge transfer. The coupling layers are bijective, preserving information while mapping features into a shared canonical space. This invertibility avoids representational collapse and enables alignment across structurally different tasks without reducing expressive capacity. We first evaluate NexusFlow on the core challenge of domain-partitioned autonomous driving, where dense map reconstruction and sparse multi-object tracking are supervised in different geographic regions, creating both structural disparity and a strong domain gap. NexusFlow sets a new state-of-the-art result on nuScenes, outperforming strong partially supervised baselines. To demonstrate generality, we further test NexusFlow on NYUv2 using three homogeneous dense prediction tasks, segmentation, depth, and surface normals, as a representative N-task PS-MTL scenario. NexusFlow yields consistent gains across all tasks, confirming its broad applicability.