A New PHO-rmula for Improved Performance of Semi-Structured Networks

TL;DR

This paper introduces a non-invasive post-hoc orthogonalization method for semi-structured neural networks, improving model identifiability, estimation, and prediction accuracy while maintaining interpretability.

Contribution

The paper proposes a novel PHO technique that enhances semi-structured neural networks by ensuring component identifiability without compromising their interpretability.

Findings

PHO guarantees model component identifiability.

Improved estimation and prediction accuracy demonstrated.

Method outperforms existing techniques in experiments.

Abstract

Recent advances to combine structured regression models and deep neural networks for better interpretability, more expressiveness, and statistically valid uncertainty quantification demonstrate the versatility of semi-structured neural networks (SSNs). We show that techniques to properly identify the contributions of the different model components in SSNs, however, lead to suboptimal network estimation, slower convergence, and degenerated or erroneous predictions. In order to solve these problems while preserving favorable model properties, we propose a non-invasive post-hoc orthogonalization (PHO) that guarantees identifiability of model components and provides better estimation and prediction quality. Our theoretical findings are supported by numerical experiments, a benchmark comparison as well as a real-world application to COVID-19 infections.