Multi-Task Learning with Additive U-Net for Image Denoising and Classification

TL;DR

This paper introduces Additive U-Net, a novel architecture that uses additive skip fusion for improved stability and task decoupling in multi-task image denoising and classification, without increasing complexity.

Contribution

The paper proposes Additive U-Net with additive skip fusion, providing a regularization mechanism that enhances stability and task decoupling in multi-task learning.

Findings

AddUNet achieves competitive denoising performance.

Learned skip weights show task-aware redistribution.

Reconstruction remains robust with limited classification capacity.

Abstract

We investigate additive skip fusion in U-Net architectures for image denoising and denoising-centric multi-task learning (MTL). By replacing concatenative skips with gated additive fusion, the proposed Additive U-Net (AddUNet) constrains shortcut capacity while preserving fixed feature dimensionality across depth. This structural regularization induces controlled encoder-decoder information flow and stabilizes joint optimization. Across single-task denoising and joint denoising-classification settings, AddUNet achieves competitive reconstruction performance with improved training stability. In MTL, learned skip weights exhibit systematic task-aware redistribution: shallow skips favor reconstruction, while deeper features support discrimination. Notably, reconstruction remains robust even under limited classification capacity, indicating implicit task decoupling through additive fusion. These findings show that simple constraints on skip connections act as an effective architectural regularizer for stable and scalable multi-task learning without increasing model complexity.