A Cross-Perspective Annotated Dataset for Dynamic Object-Level Attention Modeling in Cloud Gaming

TL;DR

This paper introduces a new annotated dataset from GTA V gameplay that captures object semantics and relationships, aiming to improve deep learning models for dynamic object-level attention in cloud gaming.

Contribution

The paper presents a novel dataset with semantic annotations and analysis of factors influencing player interest, enhancing data resources for DL-based cloud gaming research.

Findings

Player's in-game speed affects interest levels

Object size influences attention focus

Object speed impacts player interest

Abstract

Cloud gaming has gained popularity as it provides high-quality gaming experiences on thin hardware, such as phones and tablets. Transmitting gameplay frames at high resolutions and ultra-low latency is the key to guaranteeing players' quality of experience (QoE). Numerous studies have explored deep learning (DL) techniques to address this challenge. The efficiency of these DL-based approaches is highly affected by the dataset. However, existing datasets usually focus on the positions of objects while ignoring semantic relationships with other objects and their unique features. In this paper, we present a game dataset by collecting gameplay clips from Grand Theft Auto (GTA) V, and annotating the player's interested objects during the gameplay. Based on the collected data, we analyze several factors that have an impact on player's interest and identify that the player's in-game speed, object's size, and object's speed are the main factors. The dataset is available at https://drive.google.com/drive/folders/1idH251a2K-hGGd3pKjX-3Gx5o_rUqLC4?usp=sharing