Containers for portable, productive and performant scientific computing

TL;DR

This paper evaluates Linux container technology for scientific computing, demonstrating no performance penalties on HPC systems and offering guidance for effective container deployment in scientific environments.

Contribution

It provides an empirical assessment of container performance in scientific computing and offers practical guidance for deploying containers on diverse platforms, including HPC systems.

Findings

No performance penalties when running scientific programs inside containers.

Faster Python library imports reduce runtime on large parallel computers.

Guidance for HPC vendors to optimize container integration.

Abstract

Containers are an emerging technology that hold promise for improving productivity and code portability in scientific computing. We examine Linux container technology for the distribution of a non-trivial scientific computing software stack and its execution on a spectrum of platforms from laptop computers through to high performance computing (HPC) systems. We show on a workstation and a leadership-class HPC system that when deployed appropriately there are no performance penalties running scientific programs inside containers. For Python code run on large parallel computers, the run time is reduced inside a container due to faster library imports. The software distribution approach and data that we present will help developers and users decide on whether container technology is appropriate for them. We also provide guidance for the vendors of HPC systems that rely on proprietary libraries for performance on what they can do to make containers work seamlessly and without performance penalty.