Enhanced Response Envelope via Envelope Regularization

TL;DR

This paper introduces an enhanced response envelope model with regularization that improves prediction accuracy and handles high-dimensional data better than traditional methods, especially when predictors outnumber samples.

Contribution

It proposes a novel envelope regularization technique based on a nonconvex manifold, improving prediction risk and high-dimensional applicability of the response envelope model.

Findings

Enhanced response envelope yields lower prediction risk.

The method effectively handles high-dimensional data.

Simulation and real data show improved prediction performance.

Abstract

The response envelope model provides substantial efficiency gains over the standard multivariate linear regression by identifying the material part of the response to the model and by excluding the immaterial part. In this paper, we propose the enhanced response envelope by incorporating a novel envelope regularization term based on a nonconvex manifold formulation. It is shown that the enhanced response envelope can yield better prediction risk than the original envelope estimator. The enhanced response envelope naturally handles high-dimensional data for which the original response envelope is not serviceable without necessary remedies. In an asymptotic high-dimensional regime where the ratio of the number of predictors over the number of samples converges to a non-zero constant, we characterize the risk function and reveal an interesting double descent phenomenon for the envelope model. A simulation study confirms our main theoretical findings. Simulations and real data applications demonstrate that the enhanced response envelope does have significantly improved prediction performance over the original envelope method, especially when the number of predictors is close to or moderately larger than the number of samples. Proofs and additional simulation results are shown in the supplementary file to this paper.