Neural Network Machine Regression (NNMR): A Deep Learning Framework for Uncovering High-order Synergistic Effects

TL;DR

The paper introduces NNMR, a deep learning framework that performs feature selection and function estimation, capturing high-order effects with interpretable sparse architectures and valid inference in high-dimensional settings.

Contribution

It presents NNMR, a novel neural network approach integrating input gating and adaptive regularization for interpretable high-order effect modeling and valid post-selection inference.

Findings

Outperforms existing methods in selection accuracy.

Scales efficiently to high-dimensional data.

Provides reliable inference without parametric assumptions.

Abstract

We propose a new neural network framework, termed Neural Network Machine Regression (NNMR), which integrates trainable input gating and adaptive depth regularization to jointly perform feature selection and function estimation in an end-to-end manner. By penalizing both gating parameters and redundant layers, NNMR yields sparse and interpretable architectures while capturing complex nonlinear relationships driven by high-order synergistic effects. We further develop a post-selection inference procedure based on split-sample, permutation-based hypothesis testing, enabling valid inference without restrictive parametric assumptions. Compared with existing methods, including Bayesian kernel machine regression and widely used post hoc attribution techniques, NNMR scales efficiently to high-dimensional feature spaces while rigorously controlling type I error. Simulation studies demonstrate its superior selection accuracy and inference reliability. Finally, an empirical application reveals sparse, biologically meaningful food group predictors associated with somatic growth among adolescents living in Mexico City.