Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization

TL;DR

This paper introduces an advanced multi-task learning framework for dense vision tasks that enhances cross-task coherence and dynamically balances task training, leading to state-of-the-art results on benchmark datasets.

Contribution

It proposes a novel MTL model using vision transformers, a trace-back method for coherence, and a dynamic task balancing strategy, addressing limitations of previous approaches.

Findings

Achieves new state-of-the-art performance on benchmark datasets.

Improves geometric and predictive coherence across tasks.

Effectively balances task training complexity.

Abstract

Multi-Task Learning (MTL) involves the concurrent training of multiple tasks, offering notable advantages for dense prediction tasks in computer vision. MTL not only reduces training and inference time as opposed to having multiple single-task models, but also enhances task accuracy through the interaction of multiple tasks. However, existing methods face limitations. They often rely on suboptimal cross-task interactions, resulting in task-specific predictions with poor geometric and predictive coherence. In addition, many approaches use inadequate loss weighting strategies, which do not address the inherent variability in task evolution during training. To overcome these challenges, we propose an advanced MTL model specifically designed for dense vision tasks. Our model leverages state-of-the-art vision transformers with task-specific decoders. To enhance cross-task coherence, we introduce a trace-back method that improves both cross-task geometric and predictive features. Furthermore, we present a novel dynamic task balancing approach that projects task losses onto a common scale and prioritizes more challenging tasks during training. Extensive experiments demonstrate the superiority of our method, establishing new state-of-the-art performance across two benchmark datasets. The code is available at:https://github.com/Klodivio355/MT-CP