# A Three-Level Parallelisation Scheme and Application to the Nelder-Mead Algorithm

## TL;DR

This paper introduces a three-level parallelisation scheme to enhance the efficiency of parallel algorithms, exemplified by modified Nelder-Mead methods for solving PDEs and linear systems, with a focus on load balancing and scalability.

## Contribution

It proposes a novel three-level parallelisation framework that improves scalability by incorporating less efficient algorithms at an additional level, addressing efficiency drops in traditional two-level schemes.

## Key findings

- Enhanced parallel efficiency with three-level scheme.
- Effective load balancing heuristic for large processor counts.
- Successful application to PDEs and linear systems using modified Nelder-Mead.

## Abstract

We consider a three-level parallelisation scheme. The second and third levels define a classical two-level parallelisation scheme and some load balancing algorithm is used to distribute tasks among processes. It is well-known that for many applications the efficiency of parallel algorithms of the second and third level starts to drop down after some critical parallelisation degree is reached. This weakness of the two-level template is addressed by introduction of one additional parallelisation level. As an alternative to the basic solver some new or modified algorithms are considered on this level. The idea of the proposed methodology is to increase the parallelisation degree by using less efficient algorithms in comparison with the basic solver. As an example we investigate two modified Nelder-Mead methods. For the selected application, a few partial differential equations are solved numerically on the second level, and on the third level the parallel Wang's algorithm is used to solve systems of linear equations with tridiagonal matrices. A greedy workload balancing heuristic is proposed, which is oriented to the case of a large number of available processors. The complexity estimates of the computational tasks are model-based, i.e. they use empirical computational data.