Measuring Neural Network Complexity via Effective Degrees of Freedom

TL;DR

This paper introduces a robust method called generalized degrees of freedom (GDF) to measure the complexity of neural networks with binary outcomes, providing a more stable alternative to traditional likelihood-based measures.

Contribution

The paper adapts GDF for discrete responses in neural networks and demonstrates its robustness and broad applicability compared to existing complexity measures.

Findings

GDF offers a stable assessment of neural network complexity.

GDF depends on response sensitivity, not likelihood assumptions.

GDF aligns with other measures when models fit data well.

Abstract

Quantifying the complexity of feed-forward neural networks (FFNNs) remains challenging due to their nonlinear, hierarchical structure and numerous parameters. We apply generalized degrees of freedom (GDF) to measure model complexity in FFNNs with binary outcomes, adapting the algorithm for discrete responses. We compare GDF with both the effective number of parameters derived via log-likelihood cross-validation and the null degrees of freedom of Landsittel et al. Through simulation studies and a real data analysis, we demonstrate that GDF provides a robust assessment of model complexity for neural network models, as it depends only on the sensitivity of fitted values to perturbations in the observed responses rather than on assumptions about the likelihood. In contrast, cross-validation-based estimates of model complexity and the null degrees of freedom rely on the correctness of the assumed likelihood and may exhibit substantial variability. We find that GDF, cross-validation-based measures, and null degrees of freedom yield similar assessments of model complexity only when the fitted model adequately represents the data-generating mechanism. These findings highlight GDF as a stable and broadly applicable measure of model complexity for neural networks in statistical modeling.