Jupyter as Common Technology Platform for Interactive HPC Services
Michael Milligan

TL;DR
This paper discusses how the Minnesota Supercomputing Institute uses Jupyter as a unified platform for interactive HPC services, enhancing usability and workflow development while addressing resource management challenges.
Contribution
It introduces a novel approach of integrating Jupyter notebooks with traditional HPC systems to improve user experience and workflow flexibility.
Findings
Enhanced user accessibility to HPC resources
Successful integration of Jupyter with job scheduling systems
Insights into managing resource efficiency with interactive services
Abstract
The Minnesota Supercomputing Institute has implemented Jupyterhub and the Jupyter notebook server as a general-purpose point-of-entry to interactive high performance computing services. This mode of operation runs counter to traditional job-oriented HPC operations, but offers significant advantages for ease-of-use, data exploration, prototyping, and workflow development. From the user perspective, these features bring the computing cluster nearer to parity with emerging cloud computing options. On the other hand, retreating from fully-scheduled, job-based resource allocation poses challenges for resource availability and utilization efficiency, and can involve tools and technologies outside the typical core competencies of a supercomputing center's operations staff. MSI has attempted to mitigate these challenges by adopting Jupyter as a common technology platform for interactive…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
