EigenKernel - A middleware for parallel generalized eigenvalue solvers to attain high scalability and usability

TL;DR

EigenKernel is an open-source middleware that enhances the scalability and usability of parallel generalized eigenvalue solvers, enabling optimal solver selection and performance prediction on supercomputers.

Contribution

The paper introduces EigenKernel, a middleware that integrates multiple solvers and provides performance prediction capabilities for large-scale eigenvalue computations.

Findings

EigenExa and ELPA outperform ScaLAPACK in benchmarks.

Hybrid solvers show improved performance.

Bayesian inference effectively predicts solver performance.

Abstract

An open-source middleware EigenKernel was developed for use with parallel generalized eigenvalue solvers or large-scale electronic state calculation to attain high scalability and usability. The middleware enables the users to choose the optimal solver, among the three parallel eigenvalue libraries of ScaLAPACK, ELPA, EigenExa and hybrid solvers constructed from them, according to the problem specification and the target architecture. The benchmark was carried out on the Oakforest-PACS supercomputer and reveals that ELPA, EigenExa and their hybrid solvers show better performance, when compared with pure ScaLAPACK solvers. The benchmark on the K computer is also used for discussion. In addition, a preliminary research for the performance prediction was investigated, so as to predict the elapsed time T as the function of the number of used nodes P (T=T(P)). The prediction is based on Bayesian inference using the Markov Chain Monte Carlo (MCMC) method and the test calculation indicates that the method is applicable not only to performance interpolation but also to extrapolation. Such a middleware is of crucial importance for application-algorithm-architecture co-design among the current, next-generation (exascale), and future-generation (post-Moore era) supercomputers.