FIPER: Factorized Features for Robust Image Super-Resolution and Compression

TL;DR

FIPER introduces a unified factorized feature representation that improves both image super-resolution and compression by capturing structural and multi-scale details, achieving state-of-the-art results in both tasks.

Contribution

The paper proposes a novel factorized feature approach that generalizes across low-level vision tasks, enhancing performance in super-resolution and image compression.

Findings

Achieves 204.4% PSNR improvement in super-resolution

Reduces BD-rate by 9.35% in image compression

Demonstrates broad generalizability across tasks

Abstract

In this work, we propose using a unified representation, termed Factorized Features, for low-level vision tasks, where we test on Single Image Super-Resolution (SISR) and \textbf{Image Compression}. Motivated by the shared principles between these tasks, they require recovering and preserving fine image details, whether by enhancing resolution for SISR or reconstructing compressed data for Image Compression. Unlike previous methods that mainly focus on network architecture, our proposed approach utilizes a basis-coefficient decomposition as well as an explicit formulation of frequencies to capture structural components and multi-scale visual features in images, which addresses the core challenges of both tasks. We replace the representation of prior models from simple feature maps with Factorized Features to validate the potential for broad generalizability. In addition, we further optimize the compression pipeline by leveraging the mergeable-basis property of our Factorized Features, which consolidates shared structures on multi-frame compression. Extensive experiments show that our unified representation delivers state-of-the-art performance, achieving an average relative improvement of 204.4% in PSNR over the baseline in Super-Resolution (SR) and 9.35% BD-rate reduction in Image Compression compared to the previous SOTA. Project page: https://jayisaking.github.io/FIPER/