Two-level trust-region method with random subspaces

TL;DR

This paper proposes a two-level trust-region method that combines full-space and random subspace minimizations to improve convergence speed in solving unconstrained nonlinear optimization problems, demonstrated through machine learning experiments.

Contribution

It introduces a novel two-level trust-region approach utilizing random subspaces for accelerated convergence in nonlinear optimization.

Findings

Demonstrates improved convergence speed in numerical experiments

Ensures global convergence to critical points

Effective in machine learning optimization tasks

Abstract

We introduce a two-level trust-region method (TLTR) for solving unconstrained nonlinear optimization problems. Our method uses a composite iteration step, which is based on two distinct search directions. The first search direction is obtained through minimization in the full/high-resolution space, ensuring global convergence to a critical point. The second search direction is obtained through minimization in the randomly generated subspace, which, in turn, allows for convergence acceleration. The efficiency of the proposed TLTR method is demonstrated through numerical experiments in the field of machine learning