Hierarchical Fusion and Joint Aggregation: A Multi-Level Feature Representation Method for AIGC Image Quality Assessment

TL;DR

This paper introduces a multi-level feature representation method with hierarchical fusion and joint aggregation for improved AIGC image quality assessment, capturing complex distortions across perception and semantics.

Contribution

It proposes a novel multi-level visual assessment paradigm and develops two networks, MGLF-Net and MPEF-Net, for perceptual quality and Text-to-Image correspondence evaluation.

Findings

Outperforms existing methods on benchmark datasets

Effectively captures multi-dimensional distortions

Validates the effectiveness of multi-level fusion approach

Abstract

The quality assessment of AI-generated content (AIGC) faces multi-dimensional challenges, that span from low-level visual perception to high-level semantic understanding. Existing methods generally rely on single-level visual features, limiting their ability to capture complex distortions in AIGC images. To address this limitation, a multi-level visual representation paradigm is proposed with three stages, namely multi-level feature extraction, hierarchical fusion, and joint aggregation. Based on this paradigm, two networks are developed. Specifically, the Multi-Level Global-Local Fusion Network (MGLF-Net) is designed for the perceptual quality assessment, extracting complementary local and global features via dual CNN and Transformer visual backbones. The Multi-Level Prompt-Embedded Fusion Network (MPEF-Net) targets Text-to-Image correspondence by embedding prompt semantics into the visual feature fusion process at each feature level. The fused multi-level features are then aggregated for final evaluation. Experiments on benchmarks demonstrate outstanding performance on both tasks, validating the effectiveness of the proposed multi-level visual assessment paradigm.