PennyCoder: Efficient Domain-Specific LLMs for PennyLane-Based Quantum Code Generation

TL;DR

PennyCoder is a lightweight, locally deployable quantum code generation framework that fine-tunes LLaMA 3.1-8B with domain-specific instructions, enabling efficient on-device assistance for PennyLane-based quantum programming.

Contribution

We introduce PennyCoder, a novel framework that enables local quantum code generation by fine-tuning LLaMA 3.1-8B with parameter-efficient techniques for PennyLane tasks.

Findings

Achieved 44.3% accuracy on quantum code correctness.

Outperformed base LLaMA and RAG baseline models.

Demonstrated effective on-device quantum programming assistance.

Abstract

The growing demand for robust quantum programming frameworks has unveiled a critical limitation: current large language model (LLM) based quantum code assistants heavily rely on remote APIs, introducing challenges related to privacy, latency, and excessive usage costs. Addressing this gap, we propose PennyCoder, a novel lightweight framework for quantum code generation, explicitly designed for local and embedded deployment to enable on-device quantum programming assistance without external API dependence. PennyCoder leverages a fine-tuned version of the LLaMA 3.1-8B model, adapted through parameter-efficient Low-Rank Adaptation (LoRA) techniques combined with domain-specific instruction tuning optimized for the specialized syntax and computational logic of quantum programming in PennyLane, including tasks in quantum machine learning and quantum reinforcement learning. Unlike prior work focused on cloud-based quantum code generation, our approach emphasizes device-native operability while maintaining high model efficacy. We rigorously evaluated PennyCoder over a comprehensive quantum programming dataset, achieving 44.3% accuracy with our fine-tuned model (compared to 33.7% for the base LLaMA 3.1-8B and 40.1% for the RAG-augmented baseline), demonstrating a significant improvement in functional correctness.