3DGS-VBench: A Comprehensive Video Quality Evaluation Benchmark for 3DGS Compression

TL;DR

This paper introduces 3DGS-VBench, a large-scale benchmark dataset for evaluating the quality of 3D Gaussian Splatting compression methods, facilitating research in compression efficiency and visual quality assessment.

Contribution

It establishes a comprehensive VQA dataset with annotations, benchmarks multiple compression algorithms, and evaluates various quality metrics for 3DGS compression.

Findings

Benchmarking 6 compression algorithms on storage and quality

Evaluation of 15 quality assessment metrics

Validated dataset reliability with MOS scores

Abstract

3D Gaussian Splatting (3DGS) enables real-time novel view synthesis with high visual fidelity, but its substantial storage requirements hinder practical deployment, prompting state-of-the-art (SOTA) 3DGS methods to incorporate compression modules. However, these 3DGS generative compression techniques introduce unique distortions lacking systematic quality assessment research. To this end, we establish 3DGS-VBench, a large-scale Video Quality Assessment (VQA) Dataset and Benchmark with 660 compressed 3DGS models and video sequences generated from 11 scenes across 6 SOTA 3DGS compression algorithms with systematically designed parameter levels. With annotations from 50 participants, we obtained MOS scores with outlier removal and validated dataset reliability. We benchmark 6 3DGS compression algorithms on storage efficiency and visual quality, and evaluate 15 quality assessment metrics across multiple paradigms. Our work enables specialized VQA model training for 3DGS, serving as a catalyst for compression and quality assessment research. The dataset is available at https://github.com/YukeXing/3DGS-VBench.