A Robust Asynchronous Newton Method for Massive Scale Computing Systems

TL;DR

This paper introduces a scalable asynchronous Newton method for distributed optimization on volunteer computing grids, demonstrating faster convergence and robustness to node unreliability compared to traditional methods.

Contribution

It extends the FGDO framework with an asynchronous Newton method, enhancing scalability and fault tolerance for large-scale distributed optimization.

Findings

ANM converges faster than conjugate gradient descent.

The method is resilient to heterogeneous and unreliable nodes.

Preliminary results show significant speedup in convergence.

Abstract

Volunteer computing grids offer super-computing levels of computing power at the relatively low cost of operating a server. In previous work, the authors have shown that it is possible to take traditionally iterative evolutionary algorithms and execute them on volunteer computing grids by performing them asynchronously. The asynchronous implementations dramatically increase scalability and decrease the time taken to converge to a solution. Iterative and asynchronous optimization algorithms implemented using MPI on clusters and supercomputers, and BOINC on volunteer computing grids have been packaged together in a framework for generic distributed optimization (FGDO). This paper presents a new extension to FGDO for an asynchronous Newton method (ANM) for local optimization. ANM is resilient to heterogeneous, faulty and unreliable computing nodes and is extremely scalable. Preliminary results show that it can converge to a local optimum significantly faster than conjugate gradient descent does.