The Landscape and Challenges of HPC Research and LLMs
Le Chen, Nesreen K. Ahmed, Akash Dutta, Arijit Bhattacharjee, Sixing, Yu, Quazi Ishtiaque Mahmud, Waqwoya Abebe, Hung Phan, Aishwarya Sarkar,, Branden Butler, Niranjan Hasabnis, Gal Oren, Vy A. Vo, Juan Pablo Munoz,, Theodore L. Willke, Tim Mattson, Ali Jannesari

TL;DR
This paper explores how large language models and high-performance computing can be integrated, emphasizing the potential benefits and adaptations needed for HPC tasks, based on recent advances in LLMs and exascale computing.
Contribution
It proposes adapting LLM techniques for HPC applications and discusses how existing ideas can be improved for high-performance computing tasks.
Findings
LLMs have revolutionized NLP and coding tasks.
HPC has achieved exascale performance levels.
Adapting LLMs for HPC can enhance computational efficiency.
Abstract
Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning. Both encoder-decoder models and prompt-based techniques have shown immense potential for natural language processing and code-based tasks. Over the past several years, many research labs and institutions have invested heavily in high-performance computing, approaching or breaching exascale performance levels. In this paper, we posit that adapting and utilizing such language model-based techniques for tasks in high-performance computing (HPC) would be very beneficial. This study presents our reasoning behind the aforementioned position and highlights how existing ideas can be improved and adapted for HPC tasks.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Data Storage Technologies · Scientific Computing and Data Management
