A Note On Nonlinear Regression Under L2 Loss

TL;DR

This paper demonstrates that a convex nonlinear regression model can be constructed for the traditional least squares problem, potentially simplifying the training process for complex systems.

Contribution

It introduces a convex nonlinear regression model under L2 loss, addressing non-convexity issues in traditional models.

Findings

Existence of convex nonlinear regression models for L2 loss

Potential for easier training of complex systems

Advancement in nonlinear regression theory

Abstract

We investigate the nonlinear regression problem under L2 loss (square loss) functions. Traditional nonlinear regression models often result in non-convex optimization problems with respect to the parameter set. We show that a convex nonlinear regression model exists for the traditional least squares problem, which can be a promising towards designing more complex systems with easier to train models.