HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts

TL;DR

HiPrompt introduces a hierarchical, tuning-free prompting method for high-resolution image generation that combines global and local guidance to improve structural quality and reduce artifacts in 4K and higher resolutions.

Contribution

The paper proposes HiPrompt, a novel hierarchical prompting framework that enhances high-resolution image generation without additional tuning, effectively addressing object repetition and structural artifacts.

Findings

Outperforms state-of-the-art methods in 4K image generation

Reduces object repetition and structural artifacts

Maintains coherent local and global semantics

Abstract

The potential for higher-resolution image generation using pretrained diffusion models is immense, yet these models often struggle with issues of object repetition and structural artifacts especially when scaling to 4K resolution and higher. We figure out that the problem is caused by that, a single prompt for the generation of multiple scales provides insufficient efficacy. In response, we propose HiPrompt, a new tuning-free solution that tackles the above problems by introducing hierarchical prompts. The hierarchical prompts offer both global and local guidance. Specifically, the global guidance comes from the user input that describes the overall content, while the local guidance utilizes patch-wise descriptions from MLLMs to elaborately guide the regional structure and texture generation. Furthermore, during the inverse denoising process, the generated noise is decomposed into low- and high-frequency spatial components. These components are conditioned on multiple prompt levels, including detailed patch-wise descriptions and broader image-level prompts, facilitating prompt-guided denoising under hierarchical semantic guidance. It further allows the generation to focus more on local spatial regions and ensures the generated images maintain coherent local and global semantics, structures, and textures with high definition. Extensive experiments demonstrate that HiPrompt outperforms state-of-the-art works in higher-resolution image generation, significantly reducing object repetition and enhancing structural quality.