TROPHY: Trust Region Optimization Using a Precision Hierarchy

TL;DR

This paper introduces TROPHY, a trust-region optimization algorithm that leverages mixed-precision evaluations to enhance computational efficiency and reduce resource consumption in large-scale nonlinear problems.

Contribution

The paper proposes a novel trust-region method that adaptively uses different floating-point precisions, demonstrating potential energy and cost savings in high-performance computing applications.

Findings

Mixed-precision evaluations reduce computational load.

Significant energy savings possible with hardware implementation.

Effective on climate models and large-scale data assimilation.

Abstract

We present an algorithm to perform trust-region-based optimization for nonlinear unconstrained problems. The method selectively uses function and gradient evaluations at different floating-point precisions to reduce the overall energy consumption, storage, and communication costs; these capabilities are increasingly important in the era of exascale computing. In particular, we are motivated by a desire to improve computational efficiency for massive climate models. We employ our method on two examples: the CUTEst test set and a large-scale data assimilation problem to recover wind fields from radar returns. Although this paper is primarily a proof of concept, we show that if implemented on appropriate hardware, the use of mixed-precision can significantly reduce the computational load compared with fixed-precision solvers.