QiboAgent: a practitioner's guideline to open source assistants for Quantum Computing code development

TL;DR

QiboAgent is an open-source framework that uses hybrid LLM workflows to improve quantum software development, reducing hallucinations and automating complex coding tasks.

Contribution

It introduces a hybrid LLM approach with retrieval and autonomous workflows for quantum software, demonstrating improved accuracy and automation capabilities.

Findings

Achieved 90.2% code generation accuracy with small open-source models.

Reduced hallucination rates compared to proprietary models.

Automated complex software engineering tasks like refactoring and language bindings.

Abstract

We introduce QiboAgent, a reference implementation designed to serve as a practitioner's guideline for developing specialized coding assistants in Quantum Computing middleware. Addressing the limitations in scientific software development of general-purpose proprietary models, we explore how lightweight, open-source Large Language Models (LLMs) provided with a custom workflow architecture compare. In detail, we experiment with two complementary paradigms: a Retrieval-Augmented Generation pipeline for high-precision information retrieval, and an autonomous agentic workflow for complex software engineering tasks. We observe that this hybrid approach significantly reduces hallucination rates in code generation compared to a proprietary baseline, achieving a peak accuracy of 90.2% with relatively small open-source models of size up to 30B parameters. Furthermore, the agentic framework exhibits advanced coding capabilities, automating the resolution of maintenance issues and new features requests, or by prototyping larger-scale refactors of the codebase, such as producing a compiled Rust module with bindings of an original pure python package, Qibo in our case. The LLM workflows used for our analysis are integrated into a user interface and a Model Context Protocol server, providing an accessible tool for Qibo developers.