Estimation from Non-Linear Observations via Convex Programming with Application to Bilinear Regression

TL;DR

This paper introduces a convex programming estimator for non-linear regression problems with difference of convex functions, extending anchored regression, and demonstrates its effectiveness in bilinear regression with theoretical guarantees.

Contribution

It develops a new convex estimator for broad non-linear regression models, including bilinear regression, with provable accuracy and sample complexity bounds, and provides a method to construct the necessary approximation oracle.

Findings

Estimator achieves high-probability accurate recovery

Sample complexity bounds are established for bilinear regression

A tractable scheme for constructing the approximation oracle is proposed

Abstract

We propose a computationally efficient estimator, formulated as a convex program, for a broad class of non-linear regression problems that involve difference of convex (DC) non-linearities. The proposed method can be viewed as a significant extension of the "anchored regression" method formulated and analyzed in [10] for regression with convex non-linearities. Our main assumption, in addition to other mild statistical and computational assumptions, is availability of a certain approximation oracle for the average of the gradients of the observation functions at a ground truth. Under this assumption and using a PAC-Bayesian analysis we show that the proposed estimator produces an accurate estimate with high probability. As a concrete example, we study the proposed framework in the bilinear regression problem with Gaussian factors and quantify a sufficient sample complexity for exact recovery. Furthermore, we describe a computationally tractable scheme that provably produces the required approximation oracle in the considered bilinear regression problem.