QuIC: Quantum-Inspired Compound Adapters for Parameter Efficient Fine-Tuning
Snehal Raj, Brian Coyle
TL;DR
QuIC adapters are a quantum-inspired, highly memory-efficient method for fine-tuning large models, enabling substantial parameter compression with minimal performance loss, suitable for resource-limited settings.
Contribution
Introduction of QuIC adapters, a novel PEFT method inspired by quantum circuits that significantly reduces memory footprint and enables effective model fine-tuning.
Findings
QuIC adapters achieve over 40x parameter compression compared to LoRA.
QuIC adapters match performance of existing orthogonal methods in first-order mode.
Higher-order QuIC configurations enable substantial compression with modest performance trade-offs.
Abstract
Scaling full finetuning of large foundation models strains GPU memory and training time. Parameter Efficient Fine-Tuning (PEFT) methods address this issue via adapter modules which update only a small subset of model parameters. In this work, we introduce Quantum-Inspired Compound Adapters (QuIC Adapters), a PEFT approach inspired from Hamming-weight preserving quantum circuits that can effectively finetune a model using less than 0.02\% memory footprint of the base model. QuIC adapters preserve pretrained representations by enforcing orthogonality in weight parameters, and have native deployment mechanisms on quantum computers. We test QuIC adapters by finetuning large language models like LLaMA and vision transformers on language, math, reasoning and vision benchmarks. In its first-order configuration, QuIC recovers the performance of existing orthogonal methods, while higher-order…
Peer Reviews
Decision·Submitted to ICLR 2026
+ The orthogonality-preserving property of compound matrices is rigorously proved via the Cauchy–Binet theorem, and the resulting adapter formulation is mathematically coherent. The method reduces cleanly to OFT when using only first-order compounds, providing a clear conceptual lineage. + The determinantal compound mechanism introduces a distinct parameterization not explored in prior PEFT work (LoRA, OFT, BOFT). The “quantum-inspired” construction offers a compact and theoretically elegant wa
- The main NLP evaluation is on GLUE, which is now a saturated and outdated benchmark for PEFT methods. Modern evaluations typically use SuperGLUE, MMLU, BIG-Bench Hard, or instruction-tuning datasets to assess adaptation quality in LLMs. As a result, the GLUE results provide limited evidence of QuIC’s effectiveness for large-scale or reasoning-centric fine-tuning. - While the compound adapter design is original, the overall mechanism—orthogonality-preserving multiplicative fine-tuning—extends
* The paper tests multiple baselines across language and vision models and datasets. * They clearly mention their similarities to previous orthogonal adapters; for example, recovering OFT under their framework. * They demonstrate better Pareto scores for their adapter (particularly, it often requires fewer parameters than other methods).
* The frontier (Fig. 4.a) is constructed from single configuration points for each baseline rather than full \emph{parameter-budget sweeps} (e.g., LoRA ranks, (B)OFT block sizes). When tracing each method’s own curve, the frontier could shift significantly. * Results mostly compare against much larger adapters (in # of parameters). For a closer apples-to-apples comparison (in terms of benchmark accuracy as opposed to Parameter score), it would be nice to see how QuIC compares when parameter-mat
1. The construction of compound matrices from Hamming-weight preserving quantum circuits provides an original mathematical and conceptual bridge between quantum computing and PEFT. 2. Achieves remarkable compression (<0.02% memory of base model) with minimal accuracy loss, demonstrating practicality for low-resource or edge deployment. 3. Evaluations span multiple modalities (language, vision, math, reasoning) and strong baselines (LoRA, OFT, BOFT, AdaLoRA, QuanTA). 4. The paper provides formal
1. The intuition connecting Hamming-weight preservation to improved fine-tuning efficiency is conceptually appealing but not rigorously shown to yield empirical benefits beyond orthogonality constraints. The determinant-based compounding operation lacks a clear learning-theoretic explanation. 2. On most GLUE and MATH10K tasks, QuIC underperforms baseline PEFT methods in raw accuracy. The paper leans heavily on Pareto efficiency to claim superiority; this metric, while interesting, does not full
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSeismic Imaging and Inversion Techniques · Magneto-Optical Properties and Applications · Photonic and Optical Devices
MethodsAdapter