Benchmarking the Lights Out Problem on Real Quantum Hardware

TL;DR

This paper benchmarks the Lights Out problem on real quantum hardware, evaluating Grover's search performance on IBM and IQM devices with insights into hardware improvements, device reliability, and calibration effects.

Contribution

It provides the first real hardware benchmarking of Lights Out problem instances and analyzes hardware performance variations and calibration impacts.

Findings

IBM hardware improved from Heron r1 to r2

IQM devices produced near-uniform output distributions

Device performance varies significantly even within the same manufacturing revision

Abstract

We implement the Lights Out problem on a 2D grid and on Mobius ladder graphs and evaluate the performance of Grover's search on real quantum hardware. We use two instances using 9 and 16 qubits, and implement them on publicly available quantum hardware by IBM and IQM. Our experiments show improvements in IBM hardware between the Heron r1 and Heron r2 generations, highlighting progress in IBM hardware during the 2023-2024 period. The Lights Out circuits produced output distributions close to uniform on IQM devices. To diagnose device limitations, we additionally ran a small Grover SAT baseline, finding that IQM Garnet performs more reliably than other tested IQM devices. We also observed that QPUs of the same manufacturing revision can differ significantly in performance (a newer device is not guaranteed to be better), and that calibration has a significant impact on the performance of quantum devices, so the choice of device strongly depends on calibration quality.