Autoregressive Image Generation with Vision Full-view Prompt

TL;DR

This paper introduces a novel Vision Full-view prompt (VF prompt) for autoregressive image generation, inspired by NLP prompt engineering, which improves image structure reconstruction, stability, and overall performance by simulating human visual perception processes.

Contribution

The paper proposes the VF prompt technique for AR image generation, enhancing contextual understanding and stability, and achieving approximately 20% performance improvement over previous methods.

Findings

20% performance improvement with VF prompts

Enhanced image structure reconstruction and stability

Better alignment with human visual perception processes

Abstract

In autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive biases. However, directly applying LLMs to complex image generation can struggle with reconstructing the image's structure and details, impacting the generation's accuracy and stability. Additionally, the 'next-token prediction' paradigm in the AR model does not align with the contextual scanning and logical reasoning processes involved in human visual perception, limiting effective image generation. Prompt engineering, as a key technique for guiding LLMs, leverages specifically designed prompts to improve model performance on complex natural language processing (NLP) tasks, enhancing accuracy and stability of generation while maintaining contextual coherence and logical consistency, similar to human reasoning. Inspired by prompt engineering from the field of NLP, we propose Vision Full-view prompt (VF prompt) to enhance autoregressive image generation. Specifically, we design specialized image-related VF prompts for AR image generation to simulate the process of human image creation. This enhances contextual logic ability by allowing the model to first perceive overall distribution information before generating the image, and improve generation stability by increasing the inference steps. Compared to the AR method without VF prompts, our method shows outstanding performance and achieves an approximate improvement of 20%.