Feature-Function Curvature Analysis: A Geometric Framework for Explaining Differentiable Models

TL;DR

This paper introduces Feature-Function Curvature Analysis (FFCA), a geometric framework that quantifies feature roles and their evolution during training, offering deeper insights into model learning and diagnostics.

Contribution

The paper presents FFCA, a novel geometric framework that captures feature impact, volatility, non-linearity, and interactions, including a dynamic extension to track learning evolution.

Findings

Models learn simple effects before complex interactions.

Dynamic analysis predicts overfitting onset.

FFCA provides geometric context for trustworthy explanations.

Abstract

Explainable AI (XAI) is critical for building trust in complex machine learning models, yet mainstream attribution methods often provide an incomplete, static picture of a model's final state. By collapsing a feature's role into a single score, they are confounded by non-linearity and interactions. To address this, we introduce Feature-Function Curvature Analysis (FFCA), a novel framework that analyzes the geometry of a model's learned function. FFCA produces a 4-dimensional signature for each feature, quantifying its: (1) Impact, (2) Volatility, (3) Non-linearity, and (4) Interaction. Crucially, we extend this framework into Dynamic Archetype Analysis, which tracks the evolution of these signatures throughout the training process. This temporal view moves beyond explaining what a model learned to revealing how it learns. We provide the first direct, empirical evidence of hierarchical learning, showing that models consistently learn simple linear effects before complex interactions. Furthermore, this dynamic analysis provides novel, practical diagnostics for identifying insufficient model capacity and predicting the onset of overfitting. Our comprehensive experiments demonstrate that FFCA, through its static and dynamic components, provides the essential geometric context that transforms model explanation from simple quantification to a nuanced, trustworthy analysis of the entire learning process.