Interaction Pursuit Biconvex Optimization

TL;DR

This paper introduces IPBO, a novel biconvex optimization method for multivariate regression that effectively handles high-dimensional data with complex correlation structures, providing theoretical guarantees and superior empirical performance.

Contribution

The paper proposes a new structured sparsity penalty-based method, IPBO, for high-dimensional multivariate regression with complex correlations, including theoretical analysis and efficient algorithms.

Findings

IPBO outperforms existing methods in simulations.

IPBO effectively captures complex correlation structures.

Theoretical guarantees support the method's reliability.

Abstract

Multivariate regression models are widely used in various fields such as biology and finance. In this paper, we focus on two key challenges: (a) When should we favor a multivariate model over a series of univariate models; (b) If the numbers of responses and predictors are allowed to greatly exceed the sample size, how to reduce the computational cost and provide precise estimation. The proposed method, Interaction Pursuit Biconvex Optimization (IPBO), explores the regression relationship allowing the predictors and responses derived from different multivariate normal distributions with general covariance matrices. In practice, the correlation structures within are complex and interact on each other based on the regression function. The proposed method solves this problem by building a structured sparsity penalty to encourages the shared structure between the network and the regression coefficients. We prove theoretical results under interpretable conditions, and provide an efficient algorithm to compute the estimator. Simulation studies and real data examples compare the proposed method with several existing methods, indicating that IPBO works well.