Tensor Product Neural Networks for Functional ANOVA Model

TL;DR

This paper introduces ANOVA-TPNN, a neural network that guarantees a unique and stable functional ANOVA decomposition, improving interpretability and estimation stability for high-dimensional functions.

Contribution

The paper proposes a novel neural network model, ANOVA-TPNN, that ensures a unique functional ANOVA decomposition for stable and accurate component estimation.

Findings

ANOVA-TPNN can approximate any smooth function effectively.

It provides more stable component estimation than existing neural networks.

The model enhances interpretability of high-dimensional functions.

Abstract

Interpretability for machine learning models is becoming more and more important as machine learning models become more complex. The functional ANOVA model, which decomposes a high-dimensional function into a sum of lower dimensional functions (commonly referred to as components), is one of the most popular tools for interpretable AI, and recently, various neural networks have been developed for estimating each component in the functional ANOVA model. However, such neural networks are highly unstable when estimating each component since the components themselves are not uniquely defined. That is, there are multiple functional ANOVA decompositions for a given function. In this paper, we propose a novel neural network which guarantees a unique functional ANOVA decomposition and thus is able to estimate each component stably and accurately. We call our proposed neural network ANOVA Tensor Product Neural Network (ANOVA-TPNN) since it is motivated by the tensor product basis expansion. Theoretically, we prove that ANOVA-TPNN can approximate any smooth function well. Empirically, we show that ANOVA-TPNN provide much more stable estimation of each component and thus much more stable interpretation when training data and initial values of the model parameters vary than existing neural networks do.