Similarity Trajectories: Linking Sampling Process to Artifacts in Diffusion-Generated Images

TL;DR

This paper introduces the concept of Similarity Trajectory to link the sampling process in diffusion models with artifact severity, enabling artifact detection with minimal annotated data.

Contribution

It proposes a novel trajectory-based method for artifact detection in diffusion-generated images, reducing the need for extensive annotated datasets.

Findings

Classifier achieves 72.35% accuracy in artifact detection.

Similarity Trajectory correlates with artifact severity.

Method requires only 0.1% of prior annotated data.

Abstract

Artifact detection algorithms are crucial to correcting the output generated by diffusion models. However, because of the variety of artifact forms, existing methods require substantial annotated data for training. This requirement limits their scalability and efficiency, which restricts their wide application. This paper shows that the similarity of denoised images between consecutive time steps during the sampling process is related to the severity of artifacts in images generated by diffusion models. Building on this observation, we introduce the concept of Similarity Trajectory to characterize the sampling process and its correlation with the image artifacts presented. Using an annotated data set of 680 images, which is only 0.1% of the amount of data used in the prior work, we trained a classifier on these trajectories to predict the presence of artifacts in images. By performing 10-fold validation testing on the balanced annotated data set, the classifier can achieve an accuracy of 72.35%, highlighting the connection between the Similarity Trajectory and the occurrence of artifacts. This approach enables differentiation between artifact-exhibiting and natural-looking images using limited training data.