Modular Jets for Supervised Pipelines: Diagnosing Mirage vs Identifiability

TL;DR

This paper introduces Modular Jets, a method to diagnose whether the internal structure of supervised models is uniquely identifiable or indistinguishable due to mirage regimes, enhancing interpretability beyond predictive performance.

Contribution

It proposes Modular Jets for analyzing model decompositions, defines mirage and identifiable regimes, and provides theoretical guarantees and an algorithm for empirical jet estimation and diagnostics.

Findings

Identified conditions for jet identifiability in linear regression pipelines.

Demonstrated the framework on linear and deep models.

Provided an algorithm for empirical jet estimation and mirage detection.

Abstract

Classical supervised learning evaluates models primarily via predictive risk on hold-out data. Such evaluations quantify how well a function behaves on a distribution, but they do not address whether the internal decomposition of a model is uniquely determined by the data and evaluation design. In this paper, we introduce \emph{Modular Jets} for regression and classification pipelines. Given a task manifold (input space), a modular decomposition, and access to module-level representations, we estimate empirical jets, which are local linear response maps that describe how each module reacts to small structured perturbations of the input. We propose an empirical notion of \emph{mirage} regimes, where multiple distinct modular decompositions induce indistinguishable jets and thus remain observationally equivalent, and contrast this with an \emph{identifiable} regime, where the observed jets single out a decomposition up to natural symmetries. In the setting of two-module linear regression pipelines we prove a jet-identifiability theorem. Under mild rank assumptions and access to module-level jets, the internal factorisation is uniquely determined, whereas risk-only evaluation admits a large family of mirage decompositions that implement the same input-to-output map. We then present an algorithm (MoJet) for empirical jet estimation and mirage diagnostics, and illustrate the framework using linear and deep regression as well as pipeline classification.