What Makes a Level Hard in Super Mario Maker 2?

TL;DR

This study analyzes user-created levels in Super Mario Maker 2 to identify factors influencing level difficulty, using statistical and NLP methods, providing insights into design elements that affect player experience.

Contribution

It introduces a comprehensive analysis combining regression and NLP techniques to understand what makes levels in SMM2 difficult, highlighting key design factors.

Findings

Level style, popularity, and timing influence difficulty.

Topics and sentiments correlate with level difficulty.

Key differences between easy and hard levels are identified.

Abstract

Games like Super Mario Maker 2 (SMM2) lower the barrier for casual users to become level designers. In this paper, we set out to analyze a vast amount of data about SMM2 user-written levels, in order to understand what factors affect a level's difficulty as experienced by other users. To this end, we perform two kinds of analyses: one based on regression models and one using natural language processing techniques. The main results shed light on which level characteristics (e.g., its style, popularity, timing) and which topics and sentiments have a consistent association with easier or harder levels. While none of our findings are startling, they help distill some key differences between easy and hard SMM2 levels, which, in turn, can pave the way for a better understanding of end-user level design.