A Novel Image Similarity Metric for Scene Composition Structure

TL;DR

This paper introduces SCSSIM, a new analytical metric that accurately measures the preservation of scene composition structure in images, addressing limitations of existing methods and aiding generative AI evaluation.

Contribution

The paper presents SCSSIM, a training-free, statistically-based metric that effectively assesses structural fidelity of scene composition in images, outperforming traditional and neural network-based metrics.

Findings

SCSSIM is invariant to non-structural distortions.

It accurately detects changes in scene composition structure.

Outperforms existing metrics in structural evaluation tasks.

Abstract

The rapid advancement of generative AI models necessitates novel methods for evaluating image quality that extend beyond human perception. A critical concern for these models is the preservation of an image's underlying Scene Composition Structure (SCS), which defines the geometric relationships among objects and the background, their relative positions, sizes, orientations, etc. Maintaining SCS integrity is paramount for ensuring faithful and structurally accurate GenAI outputs. Traditional image similarity metrics often fall short in assessing SCS. Pixel-level approaches are overly sensitive to minor visual noise, while perception-based metrics prioritize human aesthetic appeal, neither adequately capturing structural fidelity. Furthermore, recent neural-network-based metrics introduce training overheads and potential generalization issues. We introduce the SCS Similarity Index Measure (SCSSIM), a novel, analytical, and training-free metric that quantifies SCS preservation by exploiting statistical measures derived from the Cuboidal hierarchical partitioning of images, robustly capturing non-object-based structural relationships. Our experiments demonstrate SCSSIM's high invariance to non-compositional distortions, accurately reflecting unchanged SCS. Conversely, it shows a strong monotonic decrease for compositional distortions, precisely indicating when SCS has been altered. Compared to existing metrics, SCSSIM exhibits superior properties for structural evaluation, making it an invaluable tool for developing and evaluating generative models, ensuring the integrity of scene composition.