RobuMTL: Enhancing Multi-Task Learning Robustness Against Weather Conditions

TL;DR

RobuMTL is a new multi-task learning architecture that adaptively enhances robustness to adverse weather conditions by dynamically selecting specialized modules, significantly improving performance on real-world datasets.

Contribution

Introduces RobuMTL, a novel adaptive multi-task learning framework utilizing hierarchical LoRA modules and a mixture-of-experts approach for robustness against weather-induced visual degradation.

Findings

+2.8% average improvement on PASCAL under perturbations

+44.4% improvement under mixed weather conditions

+9.7% average improvement on NYUD-v2

Abstract

Robust Multi-Task Learning (MTL) is crucial for autonomous systems operating in real-world environments, where adverse weather conditions can severely degrade model performance and reliability. In this paper, we introduce RobuMTL, a novel architecture designed to adaptively address visual degradation by dynamically selecting task-specific hierarchical Low-Rank Adaptation (LoRA) modules and a LoRA expert squad based on input perturbations in a mixture-of-experts fashion. Our framework enables adaptive specialization based on input characteristics, improving robustness across diverse real-world conditions. To validate our approach, we evaluated it on the PASCAL and NYUD-v2 datasets and compared it against single-task models, standard MTL baselines, and state-of-the-art methods. On the PASCAL benchmark, RobuMTL delivers a +2.8% average relative improvement under single perturbations and up to +44.4% under mixed weather conditions compared to the MTL baseline. On NYUD-v2, RobuMTL achieves a +9.7% average relative improvement across tasks. The code is available at GitHub.