Exploring LLM-Driven Explanations for Quantum Algorithms

TL;DR

This paper investigates how large language models can support understanding of quantum algorithms by analyzing explanation quality, consistency, and improvement capabilities across different models and prompt styles.

Contribution

It provides the first empirical comparison of LLMs' effectiveness in explaining quantum algorithms and explores methods to enhance explanation quality and consistency.

Findings

Llama2 offers the best explanations from scratch.

Gpt3.5 is most effective at improving existing explanations.

Adding context to prompts significantly enhances explanation quality.

Abstract

Background: Quantum computing is a rapidly growing new programming paradigm that brings significant changes to the design and implementation of algorithms. Understanding quantum algorithms requires knowledge of physics and mathematics, which can be challenging for software developers. Aims: In this work, we provide a first analysis of how LLMs can support developers' understanding of quantum code. Method: We empirically analyse and compare the quality of explanations provided by three widely adopted LLMs (Gpt3.5, Llama2, and Tinyllama) using two different human-written prompt styles for seven state-of-the-art quantum algorithms. We also analyse how consistent LLM explanations are over multiple rounds and how LLMs can improve existing descriptions of quantum algorithms. Results: Llama2 provides the highest quality explanations from scratch, while Gpt3.5 emerged as the LLM best suited to improve existing explanations. In addition, we show that adding a small amount of context to the prompt significantly improves the quality of explanations. Finally, we observe how explanations are qualitatively and syntactically consistent over multiple rounds. Conclusions: This work highlights promising results, and opens challenges for future research in the field of LLMs for quantum code explanation. Future work includes refining the methods through prompt optimisation and parsing of quantum code explanations, as well as carrying out a systematic assessment of the quality of explanations.