GameScope: A Multi-Attribute, Multi-Codec Benchmark Dataset for Gaming Video Quality Assessment

TL;DR

This paper introduces GameScope, the largest gaming video quality dataset covering multiple codecs and content types, enabling improved quality assessment models for gaming videos.

Contribution

It provides a comprehensive, diverse dataset with annotations for gaming videos across three major codecs, including quality attributes and a benchmark for assessment methods.

Findings

Vision language model outperforms existing quality assessment methods.

Dataset includes 4,048 videos with 37 MOS ratings each.

Addresses the gap in gaming video quality assessment across multiple codecs.

Abstract

The development of video game streaming has grown rapidly, with major platforms such as YouTube and Twitch using different codecs. To support quality assessment models that work consistently across any codec, it is necessary to have access to large, diverse subjective gaming quality datasets. Currently, there are only a few available, each having limitations. To address this gap, we present the largest gaming video quality dataset to date, incorporating both user-generated content (UGC) and professional-generated content (PGC) with extensive visual diversity. Our dataset covers the most widely used codecs - H.264, H.265, and AV1 - and consists of 4,048 video samples, each annotated by an average of 37 mean opinion score (MOS) ratings. In addition to overall quality scores, we collect coarse-grained quality attributes, enabling a better understanding of perceptual factors. We study the performance of leading video quality assessment methods on this dataset, including a vision language model that outperforms all the benchmarks. To the best of our knowledge, this is the first dataset that comprehensively addresses gaming video quality assessment across multiple codecs and content types with quality attributes. Our dataset is publicly available at https://rajeshsureddi.github.io/GameScope/.