Pre-Release Experimentation in Indie Game Development: An Interview Survey

TL;DR

This paper explores how indie game developers conduct pre-release experimentation, highlighting challenges, practices, and the importance of qualitative data within resource constraints.

Contribution

It introduces a CE framework for indie game development and provides insights into experimentation practices under limited resources.

Findings

Pre-release experimentation relies mainly on qualitative data.

Resource constraints limit the scope of testing and playtesting.

Indie developers face challenges in accessing representative user data.

Abstract

[Background] The game industry faces fierce competition and games are developed on short deadlines and tight budgets. Continuously testing and experimenting with new ideas and features is essential in validating and guiding development toward market viability and success. Such continuous experimentation (CE) requires user data, which is often limited in early development stages. This challenge is further exacerbated for independent (indie) game companies with limited resources. [Aim] We wanted to gain insights into CE practices in pre-release indie game development. [Method] We performed an exploratory interview survey with 10 indie game developers from different companies and synthesised findings through an iterative coding process. [Results] We present a CE framework for game development that highlights key parts to consider when planning and implementing an experiment and note that pre-release experimentation is centred on qualitative data. Time and resource constraints impose limits on the type and extent of experimentation and playtesting that indie companies can perform, e.g. due to limited access to participants, biases and representativeness of the target audience. [Conclusions] Our results outline challenges and practices for conducting experiments with limited user data in early stages of indie game development, and may be of value also for larger game companies, and for software intensive organisations in other industries.