Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant

TL;DR

This paper introduces quantum-verifiable rewards and post-training techniques for LLMs to improve Qiskit code generation, ensuring code quality and hardware compatibility through quantum verification and synthetic data pipelines.

Contribution

It presents a novel quantum verification method and a synthetic data pipeline for aligning LLMs with quantum hardware requirements, advancing quantum-aware code generation.

Findings

Best model surpasses open-source baselines on Qiskit-HumanEval-hard

Quantum-verifiable rewards improve code quality and executability

Synthetic data pipeline supports effective model training

Abstract

Qiskit is an open-source quantum computing framework that allows users to design, simulate, and run quantum circuits on real quantum hardware. We explore post-training techniques for LLMs to assist in writing Qiskit code. We introduce quantum verification as an effective method for ensuring code quality and executability on quantum hardware. To support this, we developed a synthetic data pipeline that generates quantum problem-unit test pairs and used it to create preference data for aligning LLMs with DPO. Additionally, we trained models using GRPO, leveraging quantum-verifiable rewards provided by the quantum hardware. Our best-performing model, combining DPO and GRPO, surpasses the strongest open-source baselines on the challenging Qiskit-HumanEval-hard benchmark.