Controlled Data Rebalancing in Multi-Task Learning for Real-World Image Super-Resolution

TL;DR

This paper introduces a novel multi-task learning framework for real-world image super-resolution that balances different degradation patterns through task segmentation, imbalance quantification, and adaptive data rebalancing, leading to improved performance.

Contribution

It proposes a new task definition and imbalance quantification method, along with a data rebalancing strategy, to enhance multi-task learning for real-world image super-resolution.

Findings

Achieves superior results across various degradation tasks.

Effectively balances task contributions during training.

Demonstrates consistent qualitative and quantitative improvements.

Abstract

Real-world image super-resolution (Real-SR) is a challenging problem due to the complex degradation patterns in low-resolution images. Unlike approaches that assume a broadly encompassing degradation space, we focus specifically on achieving an optimal balance in how SR networks handle different degradation patterns within a fixed degradation space. We propose an improved paradigm that frames Real-SR as a data-heterogeneous multi-task learning problem, our work addresses task imbalance in the paradigm through coordinated advancements in task definition, imbalance quantification, and adaptive data rebalancing. Specifically, we introduce a novel task definition framework that segments the degradation space by setting parameter-specific boundaries for degradation operators, effectively reducing the task quantity while maintaining task discrimination. We then develop a focal loss based multi-task weighting mechanism that precisely quantifies task imbalance dynamics during model training. Furthermore, to prevent sporadic outlier samples from dominating the gradient optimization of the shared multi-task SR model, we strategically convert the quantified task imbalance into controlled data rebalancing through deliberate regulation of task-specific training volumes. Extensive quantitative and qualitative experiments demonstrate that our method achieves consistent superiority across all degradation tasks.