Efficient Scale-Invariant Generator with Column-Row Entangled Pixel Synthesis

TL;DR

CREPS is a novel scale-invariant generator that efficiently synthesizes high-resolution, consistent images without convolutions, using a bi-line representation for scalable and memory-efficient image generation.

Contribution

The paper introduces CREPS, a scale-equivariant generator that avoids convolutions and hierarchical designs, enabling efficient, high-resolution image synthesis with a novel bi-line feature representation.

Findings

Synthesizes images at arbitrary scales with consistency.

Operates efficiently with reduced memory footprint.

Achieves high-quality, alias-free images across datasets.

Abstract

Any-scale image synthesis offers an efficient and scalable solution to synthesize photo-realistic images at any scale, even going beyond 2K resolution. However, existing GAN-based solutions depend excessively on convolutions and a hierarchical architecture, which introduce inconsistency and the $``$texture sticking$"$ issue when scaling the output resolution. From another perspective, INR-based generators are scale-equivariant by design, but their huge memory footprint and slow inference hinder these networks from being adopted in large-scale or real-time systems. In this work, we propose $\textbf{C}$olumn-$\textbf{R}$ow $\textbf{E}$ntangled $\textbf{P}$ixel $\textbf{S}$ynthesis ($\textbf{CREPS}$), a new generative model that is both efficient and scale-equivariant without using any spatial convolutions or coarse-to-fine design. To save memory footprint and make the system scalable, we employ a novel bi-line representation that decomposes layer-wise feature maps into separate $``$thick$"$ column and row encodings. Experiments on various datasets, including FFHQ, LSUN-Church, MetFaces, and Flickr-Scenery, confirm CREPS' ability to synthesize scale-consistent and alias-free images at any arbitrary resolution with proper training and inference speed. Code is available at https://github.com/VinAIResearch/CREPS.