The Combinatorics of Falsification and Hypothesis Testing

TL;DR

This paper explores the mathematical foundations of hypothesis falsifiability, linking long-term falsifiability to topological properties and short-term falsifiability to VC finiteness, with implications for severe testing.

Contribution

It establishes a formal connection between VC finiteness, NIP structures, and short-run falsifiability, providing rigorous foundations for severe testing concepts.

Findings

VC finite hypotheses correspond to definable sets in NIP structures

Long-run falsifiability characterized by nowhere density in topology

Short-run falsifiability characterized by VC finiteness

Abstract

The present paper is concerned with the question of how falsifiable a single proposition is in the short and long run. Formal Learning theorists such as Schulte and Juhl have argued that long-run falsifiability is characterized by the topological notion of nowhere density in a suitable topological space. I argue that the short-run falsifiability of a hypothesis is in turn characterized by the VC finiteness of the hypothesis. Crucially, VC finite hypotheses correspond precisely to definable sets in NIP structures. I end the chapter by giving rigorous foundations for Mayo's conception of severe testing by way of a combinatorial, non-probabilistic notion of surprise. VC finite hypotheses again appear as the hypotheses with guaranteed short-run surprise bounds. Therefore, NIP theories and VC finite hypotheses capture the notion of short-run falsifiability.