Towards General Game Representations: Decomposing Games Pixels into Content and Style

TL;DR

This paper proposes decomposing game pixel representations into content and style embeddings using a pre-trained Vision Transformer to improve generalization across similar games for various AI tasks.

Contribution

It introduces a method to separate content and style in game representations, enhancing cross-game generalization and robustness of AI models.

Findings

Decomposed embeddings achieve style invariance across multiple games.

Content embeddings maintain strong content extraction capabilities.

Method improves generalization across game environments.

Abstract

On-screen game footage contains rich contextual information that players process when playing and experiencing a game. Learning pixel representations of games can benefit artificial intelligence across several downstream tasks including game-playing agents, procedural content generation, and player modelling. The generalizability of these methods, however, remains a challenge, as learned representations should ideally be shared across games with similar game mechanics. This could allow, for instance, game-playing agents trained on one game to perform well in similar games with no re-training. This paper explores how generalizable pre-trained computer vision encoders can be for such tasks, by decomposing the latent space into content embeddings and style embeddings. The goal is to minimize the domain gap between games of the same genre when it comes to game content critical for downstream tasks, and ignore differences in graphical style. We employ a pre-trained Vision Transformer encoder and a decomposition technique based on game genres to obtain separate content and style embeddings. Our findings show that the decomposed embeddings achieve style invariance across multiple games while still maintaining strong content extraction capabilities. We argue that the proposed decomposition of content and style offers better generalization capacities across game environments independently of the downstream task.