Photonic Quantum-Enhanced Knowledge Distillation

TL;DR

This paper introduces a hybrid quantum-classical framework called Photonic Quantum-Enhanced Knowledge Distillation (PQKD) that leverages photonic hardware randomness to improve neural network training and compression.

Contribution

PQKD integrates photonic quantum features into knowledge distillation, replacing convolutional kernels with dictionary convolutions guided by photonic measurements, enabling efficient training and compression.

Findings

Maintains high accuracy close to teacher models on simple benchmarks.

Performance degrades predictably with finite sampling, following shot-noise scaling.

Feature smoothing extends operational regime at moderate shot budgets.

Abstract

Photonic quantum processors naturally produce intrinsically stochastic measurement outcomes, offering a hardware-native source of structured randomness that can be exploited during machine-learning training. Here we introduce Photonic Quantum-Enhanced Knowledge Distillation (PQKD), a hybrid quantum photonic--classical framework in which a programmable photonic circuit generates a compact conditioning signal that constrains and guides a parameter-efficient student network during distillation from a high-capacity teacher. PQKD replaces fully trainable convolutional kernels with dictionary convolutions: each layer learns only a small set of shared spatial basis filters, while sample-dependent channel-mixing weights are derived from shot-limited photonic features and mapped through a fixed linear transform. Training alternates between standard gradient-based optimisation of the student and sampling-robust, gradient-free updates of photonic parameters, avoiding differentiation through photonic hardware. Across MNIST, Fashion-MNIST and CIFAR-10, PQKD traces a controllable compression--accuracy frontier, remaining close to teacher performance on simpler benchmarks under aggressive convolutional compression. Performance degrades predictably with finite sampling, consistent with shot-noise scaling, and exponential moving-average feature smoothing suppresses high-frequency shot-noise fluctuations, extending the practical operating regime at moderate shot budgets.