MarioNette: Self-Supervised Sprite Learning

TL;DR

MarioNette introduces a self-supervised deep learning method that decomposes sprite animations into a sparse, explicit representation of graphic elements, facilitating editing and analysis without requiring labeled data.

Contribution

The paper presents a novel self-supervised framework for decomposing sprite animations into a disentangled, dictionary-based representation, enabling easier editing and pattern discovery.

Findings

Effective decomposition of sprite animations into graphic elements.

Unsupervised discovery of recurring visual patterns.

Facilitates downstream editing and analysis tasks.

Abstract

Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision.