DeepArt: A Benchmark to Advance Fidelity Research in AI-Generated Content

TL;DR

This paper introduces DeepArt, a benchmark dataset for evaluating the fidelity of AI-generated images by GPT-4, highlighting its limitations and providing a new resource for advancing research in AI-generated content.

Contribution

It provides the first detailed analysis of GPT-4's image synthesis fidelity, revealing its limitations and establishing a new benchmark dataset for future research.

Findings

GPT-4 shows notable limitations in image synthesis fidelity.

The benchmark dataset enables systematic evaluation of AI-generated images.

The study offers both qualitative and quantitative insights into GPT-4's image generation capabilities.

Abstract

This paper explores the image synthesis capabilities of GPT-4, a leading multi-modal large language model. We establish a benchmark for evaluating the fidelity of texture features in images generated by GPT-4, comprising manually painted pictures and their AI-generated counterparts. The contributions of this study are threefold: First, we provide an in-depth analysis of the fidelity of image synthesis features based on GPT-4, marking the first such study on this state-of-the-art model. Second, the quantitative and qualitative experiments fully reveals the limitations of the GPT-4 model in image synthesis. Third, we have compiled a unique benchmark of manual drawings and corresponding GPT-4-generated images, introducing a new task to advance fidelity research in AI-generated content (AIGC). The dataset is available at: \url{https://github.com/rickwang28574/DeepArt}.