LSM: Learning Subspace Minimization for Low-level Vision

TL;DR

The paper introduces a unified learning framework called LSM that replaces heuristic regularization with learnable subspace constraints, enabling efficient multi-task low-level vision solutions with state-of-the-art performance.

Contribution

It proposes a novel LSM framework that unifies network structures for multiple low-level vision tasks, allowing shared training and generalization to unseen tasks.

Findings

Achieves state-of-the-art results on four low-level vision tasks.

Offers smaller model size and faster training convergence.

Enables real-time inference across tasks.

Abstract

We study the energy minimization problem in low-level vision tasks from a novel perspective. We replace the heuristic regularization term with a learnable subspace constraint, and preserve the data term to exploit domain knowledge derived from the first principle of a task. This learning subspace minimization (LSM) framework unifies the network structures and the parameters for many low-level vision tasks, which allows us to train a single network for multiple tasks simultaneously with completely shared parameters, and even generalizes the trained network to an unseen task as long as its data term can be formulated. We demonstrate our LSM framework on four low-level tasks including interactive image segmentation, video segmentation, stereo matching, and optical flow, and validate the network on various datasets. The experiments show that the proposed LSM generates state-of-the-art results with smaller model size, faster training convergence, and real-time inference.