Distilling the knowledge with quantum neural networks

TL;DR

This paper introduces a method to compress quantum neural networks using knowledge distillation, enabling smaller, resource-efficient models that retain performance, thus facilitating practical deployment on limited quantum hardware.

Contribution

It presents the first application of knowledge distillation to quantum neural networks, demonstrating effective compression and accelerated training convergence.

Findings

Distilled QNNs maintain accuracy with fewer qubits and shallower circuits.

Knowledge distillation reduces training costs for large-scale QNNs.

Self-distillation accelerates convergence during QNN training.

Abstract

Quantum Neural Networks (QNNs) are a promising class of quantum machine learning models with potential quantum advantages when implemented on scalable, error-corrected quantum computers. However, as system sizes increase, deploying QNNs becomes challenging. Similar to their classical counterparts, a key obstacle to their practical applications is that large-scale QNNs may not be easily deployed on smaller systems that have limited resources. Here, we tackle this challenge by compressing QNNs via knowledge distillation. We demonstrate how well-trained QNNs on large systems can be distilled into smaller architectures with similar configurations. We numerically show that knowledge distillation helps reduce the training cost of QNNs in terms of the number of qubits and circuit depth. Additionally, we find that a self-knowledge-distillation approach can accelerate training convergence. We believe our results offer new strategies for the efficient compression and practical deployment of QNNs.