Containerisation for High Performance Computing Systems: Survey and Prospects

TL;DR

This paper surveys containerisation in HPC systems, comparing it with Cloud environments, discussing current strategies, challenges, and future research directions for container orchestration in high-performance computing.

Contribution

It provides a comprehensive taxonomy of containerisation and orchestration efforts in HPC, highlighting differences with Cloud systems and identifying research challenges.

Findings

HPC containers are larger and less portable due to hardware-specific libraries.

Cloud systems have more advanced container orchestration mechanisms.

HPC container orchestration remains an emerging research area.

Abstract

Containers improve the efficiency in application deployment and thus have been widely utilised on Cloud and lately in High Performance Computing (HPC) environments. Containers encapsulate complex programs with their dependencies in isolated environments making applications more compatible and portable. Often HPC systems have higher security levels compared to Cloud systems, which restrict users' ability to customise environments. Therefore, containers on HPC need to include a heavy package of libraries making their size relatively large. These libraries usually are specifically optimised for the hardware, which compromises portability of containers. Per contra, a Cloud container has smaller volume and is more portable. Furthermore, containers would benefit from orchestrators that facilitate deployment and management of containers at a large scale. Cloud systems in practice usually incorporate sophisticated container orchestration mechanisms as opposed to HPC systems. Nevertheless, some solutions to enable container orchestration on HPC systems have been proposed in state of the art. This paper gives a survey and taxonomy of efforts in both containerisation and its orchestration strategies on HPC systems. It highlights differences thereof between Cloud and HPC. Lastly, challenges are discussed and the potentials for research and engineering are envisioned.