Neural Network Models for Contextual Regression

TL;DR

This paper introduces a simple, interpretable neural network architecture for contextual regression that effectively models context-dependent relationships with fewer parameters, outperforming traditional feed-forward networks in stability and efficiency.

Contribution

The paper presents a novel structured neural network model that separates context identification from regression, enhancing interpretability and efficiency over standard models.

Findings

The proposed SCtxtNN achieves lower mean squared error than comparable feed-forward networks.

The architecture is mathematically capable of representing contextual linear regression models.

Larger networks improve accuracy but increase complexity, while the proposed model balances performance and simplicity.

Abstract

We propose a neural network model for contextual regression in which the regression model depends on contextual features that determine the active submodel and an algorithm to fit the model. The proposed simple contextual neural network (SCtxtNN) separates context identification from context-specific regression, resulting in a structured and interpretable architecture with fewer parameters than a fully connected feed-forward network. We show mathematically that the proposed architecture is sufficient to represent contextual linear regression models using only standard neural network components. Numerical experiments are provided to support the theoretical result, showing that the proposed model achieves lower excess mean squared error and more stable performance than feed-forward neural networks with comparable numbers of parameters, while larger networks improve accuracy only at the cost of increased complexity. The results suggest that incorporating contextual structure can improve model efficiency while preserving interpretability.