Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity

TL;DR

Hi3DEval introduces a hierarchical 3D content evaluation framework that assesses object and part-level quality, including material realism, supported by a large dataset and automated scoring, outperforming existing image-based metrics.

Contribution

The paper presents Hi3DEval, a novel hierarchical evaluation framework for 3D generation quality, incorporating multi-level assessments and material realism, along with Hi3DBench dataset and automated scoring system.

Findings

Outperforms existing image-based metrics in 3D quality assessment.

Achieves better alignment with human preferences.

Provides scalable evaluation beyond manual methods.

Abstract

Despite rapid advances in 3D content generation, quality assessment for the generated 3D assets remains challenging. Existing methods mainly rely on image-based metrics and operate solely at the object level, limiting their ability to capture spatial coherence, material authenticity, and high-fidelity local details. 1) To address these challenges, we introduce Hi3DEval, a hierarchical evaluation framework tailored for 3D generative content. It combines both object-level and part-level evaluation, enabling holistic assessments across multiple dimensions as well as fine-grained quality analysis. Additionally, we extend texture evaluation beyond aesthetic appearance by explicitly assessing material realism, focusing on attributes such as albedo, saturation, and metallicness. 2) To support this framework, we construct Hi3DBench, a large-scale dataset comprising diverse 3D assets and high-quality annotations, accompanied by a reliable multi-agent annotation pipeline. We further propose a 3D-aware automated scoring system based on hybrid 3D representations. Specifically, we leverage video-based representations for object-level and material-subject evaluations to enhance modeling of spatio-temporal consistency and employ pretrained 3D features for part-level perception. Extensive experiments demonstrate that our approach outperforms existing image-based metrics in modeling 3D characteristics and achieves superior alignment with human preference, providing a scalable alternative to manual evaluations. The project page is available at https://zyh482.github.io/Hi3DEval/.