TOC-SR: Task-Optimal Compact diffusion for Image Super Resolution

TL;DR

This paper introduces TOC-SR, a compact diffusion-based framework for efficient single-step image super-resolution that significantly reduces model size and computational cost while maintaining high quality.

Contribution

The work develops a parameter-efficient diffusion backbone via architecture search and distillation, enabling fast super-resolution with minimal quality loss.

Findings

6.6x reduction in parameters compared to the original diffusion model

2.8x reduction in GMACs for the diffusion backbone

Maintains strong reconstruction quality with a single-step generator

Abstract

Diffusion models have recently demonstrated strong performance for image restoration tasks, including super-resolution. However, their large model size and iterative sampling procedures make them computationally expensive for practical deployment. In this work, we present TOC-SR, a framework for building efficient one-step super-resolution models by first discovering a compact diffusion backbone. Starting from a sixteen-channel latent diffusion model, we construct parameter-efficient surrogate blocks using feature-wise generative distillation and perform architecture discovery using epsilon-constrained Bayesian Optimization to minimize model complexity while preserving generative fidelity. The resulting compact diffusion backbone achieves a 6.6x reduction in parameters and a 2.8x reduction in GMACs compared to the expanded diffusion model. We then adapt this backbone for super-resolution and distill the diffusion process into a single-step generator. Experiments demonstrate that the proposed approach enables efficient super-resolution while maintaining strong reconstruction quality.