Pixel VQ-VAEs for Improved Pixel Art Representation

TL;DR

This paper introduces Pixel VQ-VAE, a specialized model designed to effectively learn and represent pixel art, outperforming existing models in embedding quality and downstream task performance.

Contribution

The paper presents a novel Pixel VQ-VAE model tailored for pixel art, addressing the limitations of traditional models in capturing individual pixel importance.

Findings

Outperforms other models in embedding quality

Achieves better performance on downstream tasks

Effectively captures pixel-level details in pixel art

Abstract

Machine learning has had a great deal of success in image processing. However, the focus of this work has largely been on realistic images, ignoring more niche art styles such as pixel art. Additionally, many traditional machine learning models that focus on groups of pixels do not work well with pixel art, where individual pixels are important. We propose the Pixel VQ-VAE, a specialized VQ-VAE model that learns representations of pixel art. We show that it outperforms other models in both the quality of embeddings as well as performance on downstream tasks.