On Convergence to Essential Singularities

TL;DR

This paper extends convergence analysis of iterative optimization methods to cases where the limit point may lie outside the domain of the objective function, especially for bounded rational functions near singularities.

Contribution

It proves a variant of the Łojasiewicz gradient inequality applicable near singularities of rational functions, allowing convergence results without the function being defined at the limit.

Findings

Convergence can occur even if the limit point is outside the function's domain.

A variant of the Łojasiewicz inequality is established for singularities of rational functions.

Application to analyzing divergent sequences in low-rank tensor approximation.

Abstract

An iterative optimization method applied to a function $f$ on $\mathbb{R}^n$ will produce a sequence of arguments $\{\mathbf{x}_k\}_{k \in \mathbb{N}}$; this sequence is often constrained such that $\{f(\mathbf{x}_k)\}_{k \in \mathbb{N}}$ is monotonic. As part of the analysis of an iterative method, one may ask under what conditions the sequence $\{\mathbf{x}_k\}_{k \in \mathbb{N}}$ converges. In 2005, Absil et al.\ employed the {\L}ojasiewicz gradient inequality in a proof of convergence; this requires that the objective function exist at a cluster point of the sequence. Here we provide a convergence result that does not require $f$ to be defined at the limit $\lim_{k \to \infty} \mathbf{x}_k$, should the limit exist. We show that a variant of the {\L}ojasiewicz gradient inequality holds on sets adjacent to singularities of bounded multivariate rational functions. We extend the results of Absil et al.\ to prove that if $\{\mathbf{x}_k\}_{k \in \mathbb{N}} \subset \mathbb{R}^n$ has a cluster point $\mathbf{x}_*$, if $f$ is a bounded multivariate rational function on $\mathbb{R}^n$, and if a technical condition holds, then $\mathbf{x}_k \to \mathbf{x}_*$ even if $\mathbf{x}_*$ is not in the domain of $f$. We demonstrate how this may be employed to analyze divergent sequences by mapping them to projective space, and consider the implications this has for the study of low-rank tensor approximations.