Uniqueness and stability for the solution of a nonlinear least squares problem

TL;DR

This paper investigates the uniqueness and stability of solutions to a nonlinear least squares problem, especially when the data vector does not exactly match the absolute value of a linear transformation, revealing conditions for uniqueness and stability.

Contribution

It provides the first analysis of solution uniqueness and stability for the case where the data vector differs from the absolute value of a linear transformation.

Findings

Existence of non-unique solutions for certain data vectors.

Almost all positive data vectors lead to unique solutions.

Solutions are never uniformly stable unless restricted to convex sets.

Abstract

In this paper, we focus on the nonlinear least squares: $\mbox{min}_{\mathbf{x} \in \mathbb{H}^d}\| |A\mathbf{x}|-\mathbf{b}\|$ where $A\in \mathbb{H}^{m\times d}$, $\mathbf{b} \in \mathbb{R}^m$ with $\mathbb{H} \in \{\mathbb{R},\mathbb{C} \}$ and consider the uniqueness and stability of solutions. Such problem arises, for instance, in phase retrieval and absolute value rectification neural networks. For the case where $\mathbf{b}=|A\mathbf{x}_0|$ for some $\mathbf{x}_0\in \mathbb{H}^d$, many results have been developed to characterize the uniqueness and stability of solutions. However, for the case where $\mathbf{b} \neq |A\mathbf{x}_0| $ for any $\mathbf{x}_0\in \mathbb{H}^d$, there is no existing result for it to the best of our knowledge. In this paper, we first focus on the uniqueness of solutions and show for any matrix $A\in \mathbb{H}^{m \times d}$ there always exists a vector $\mathbf{b} \in \mathbb{R}^m$ such that the solution is not unique. But, in real case, such ``bad'' vectors $\mathbf{b}$ are negligible, namely, if $\mathbf{b} \in \mathbb{R}_{+}^m$ does not lie in some measure zero set, then the solution is unique. We also present some conditions under which the solution is unique. For the stability of solutions, we prove that the solution is never uniformly stable. But if we restrict the vectors $\mathbf{b}$ to any convex set then it is stable.