A Toolkit for Measuring the Impacts of Public Funding on Open Source Software Development

TL;DR

This paper presents a comprehensive toolkit of methodological considerations and impact taxonomy to better measure and understand the effects of public funding on open source software development, addressing a key knowledge gap.

Contribution

It introduces a structured toolkit and impact taxonomy for evaluating public funding impacts on OSS, integrating prior research and practical insights.

Findings

Provides a taxonomy of impacts including social, economic, and technological effects.

Discusses qualitative, quantitative, and mixed-methods approaches for impact assessment.

Highlights considerations for regional, organizational, and project-specific factors.

Abstract

Governments are increasingly employing funding for open source software (OSS) development as a policy lever to support the security of software supply chains, digital sovereignty, economic growth, and national competitiveness in science and innovation, among others. However, the impacts of public funding on OSS development remain poorly understood, with a lack of consensus on how to meaningfully measure them. This gap hampers assessments of the return on public investment and impedes the optimisation of public-interest funding strategies. We address this gap with a toolkit of methodological considerations that may inform such measurements, drawing on prior work on OSS valuations and community health metrics by the Community Health Analytics Open Source Software (CHAOSS) project as well as our first-hand learnings as practitioners tasked with evaluating funding programmes by the Next Generation Internet initiative and the Sovereign Tech Agency. We discuss salient considerations, including the importance of accounting for funding objectives, project life stage and social structure, and regional and organisational cost factors. Next, we present a taxonomy of potential social, economic, and technological impacts that can be both positive and negative, direct and indirect, internal (i.e. within a project) and external (i.e. among a project's ecosystem of dependents and users), and manifest over various time horizons. Furthermore, we discuss the merits and limitations of qualitative, quantitative, and mixed-methods approaches, as well as options for and hazards of estimating multiplier effects. With this toolkit, we contribute to the multi-stakeholder conversation about the value and impacts of funding on OSS developers and society at large.