Generalized Factor Neural Network Model for High-dimensional Regression

TL;DR

This paper introduces a flexible neural network framework that integrates PCA-based factor modeling with non-linear transformations, improving high-dimensional regression tasks involving complex, hierarchical data structures.

Contribution

It presents a novel neural network architecture with embedded PCA and Soft PCA layers, enabling seamless factor modeling within deep learning for high-dimensional data.

Findings

Effective in simulation studies

Improves forecasting of equity ETF prices

Enhances macroeconomic nowcasting

Abstract

We tackle the challenges of modeling high-dimensional data sets, particularly those with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships. Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression. Our approach introduces PCA and Soft PCA layers, which can be embedded at any stage of a neural network architecture, allowing the model to alternate between factor modeling and non-linear transformations. This flexibility makes our method especially effective for processing hierarchical compositional data. We explore ours and other techniques for imposing low-rank structures on neural networks and examine how architectural design impacts model performance. The effectiveness of our method is demonstrated through simulation studies, as well as applications to forecasting future price movements of equity ETF indices and nowcasting with macroeconomic data.