Quantum-Enhanced Analysis and Grading of Vocal Performance

TL;DR

This paper introduces QuantumMelody, a hybrid quantum-classical system for objective singing assessment that achieves higher agreement with experts than classical methods, demonstrating feasibility for interpretable, automated vocal evaluation.

Contribution

The paper presents a novel quantum-enhanced approach for singing assessment, combining quantum circuits with classical embeddings to improve grading accuracy.

Findings

Achieves 74.29% agreement with expert graders

Outperforms classical baseline by +12.86 points

Operates efficiently on standard laptops

Abstract

We present QuantumMelody, a hybrid quantum-classical method for objective singing assessment. Grouped vocal features (pitch stability, dynamics, timbre) are encoded into a small simulated quantum circuit; all nine qubits are initialized with a Hadamard on each qubit and then receive Rx, Ry, and Rz rotations, with intra- and cross-group entanglement. The circuit measurement probabilities are fused with spectrogram transformer embeddings to estimate a grade on labels 2-5 and to surface technique-level feedback. On 168 labeled 20 second excerpts, the hybrid reaches 74.29% agreement with expert graders, a +12.86 point gain over a classical-features baseline. Processing is sub-minute per recording on a laptop-class Qiskit simulator; we do not claim hardware speedups. This is a feasibility step toward interpretable, objective singing assessment in applied audio signal processing.

Tables

Table 1. TABLE I: Features, rotation group, and empirical scaling bounds.

Feature	Rotation Group	Min	Max
Average pitch deviation (cents)	$R_x$ (pitch stability)	0	1431.7
Average jitter	$R_x$ (pitch stability)	0	0.3278
Std. dev. tempo (BPM)	$R_x$ (rhythm)	30	180
Average shimmer	$R_y$ (dynamics)	0	1.1735
Mean LUFS energy (dB)	$R_y$ (dynamics)	$-60$	$-10$
Std. dev. LUFS energy	$R_y$ (expression)	1	12
Std. dev. MFCC (timbre)	$R_z$ (timbre)	0	0.25
Zero-crossing rate	$R_z$ (clarity)	0.01	0.12
Mean tone-to-noise ratio (TNR, dB)	$R_z$ (clarity)	5	30

Equations

$$\Delta [n]=1200{\log }_2\left(\frac{F_0[n]}{F_{\mathrm{ref}}}\right).$$

$$J_{\mathrm{local}}=\frac{1}{M-1}\sum _{i=1}^{M-1}\frac{|T_{i+1}-T_i|}{T_i},$$

$$S_{\mathrm{local}}=\frac{1}{M-1}\sum _{i=1}^{M-1}\frac{|A_{i+1}-A_i|}{A_i}.$$

$$L_{\mathrm{LUFS}}=-0.691+10{\log }_{10}\left(\frac{1}{N}\sum _{n=1}^{N}w[n{]}^2\right),$$

$$E_{\mathrm{RMS}}=\sqrt{\frac{1}{N}\sum _{n=1}^{N}x[n{]}^2}.$$

$$\mathrm{TN}{\mathrm{R}}_{\mathrm{dB}}=10{\log }_{10}\left(\frac{P_{\mathrm{tone}}}{P_{\mathrm{noise}}}\right).$$

$$\mathrm{MFCC}_{k}=\sum_{m=1}^{M}\log(S_{m})\cos\!\left[\frac{\pi k}{M}\Big{(}m-\tfrac{1}{2}\Big{)}\right].$$

$$Z=\frac{1}{N-1}\sum _{n=1}^{N-1}1\{x[n]x[n+1]<0\}.$$

$$C=\frac{{\sum }_{f}f|X(f)|}{{\sum }_f|X(f)|}.$$

$$B=\sqrt{\frac{{\sum }_f(f-C{)}^2|X(f)|}{{\sum }_f|X(f)|}}.$$

$$F=\frac{\exp\!\big{(}\frac{1}{K}\sum_{k=1}^{K}\ln P_{k}\big{)}}{\frac{1}{K}\sum_{k=1}^{K}P_{k}},$$

$$E_v=1200{\log }_2\left(\frac{{\max }_n{F}_0[n]}{{\min }_n{F}_0[n]}\right),$$

$${\theta }_{\text{pitch}}$$

$${\theta }_{\text{jitter}}$$

$${\theta }_{\text{tempo}}$$

$${\theta }_{\text{shimmer}}$$

$${\theta }_{\text{LUFS}}$$

$${\theta }_{\sigma \text{LUFS}}$$

$${\theta }_{\text{MFCC}}$$

$${\theta }_{\text{ZCR}}$$

$${\theta }_{\text{TNR}}$$

$$D_{\mathrm{JS}}(P\Vert Q)=\frac{1}{2}{D}_{\mathrm{KL}}(P\Vert M)+\frac{1}{2}{D}_{\mathrm{KL}}(Q\Vert M),M=\frac{1}{2}(P+Q).$$

$$d=\sqrt{\sum_{i=1}^{n}\bigl{(}\theta_{i}^{(1)}-\theta_{i}^{(2)}\bigr{)}^{2}}.$$

Full text

I Introduction

Assessing the quality of a singing performance is a very subjective and specialized task. This is especially important in music schools, where hundreds of student recordings might need review each week and become a severe bottleneck for music teachers. The motivation for this research is rooted in the need to offer a reliable automated system for feedback with clear, consistent pointers for improvement.

Prior research in singing voice analysis provides a foundation for an objective approach. For example, humans tend to judge performances by concrete aspects such as pitch accuracy, stability of tone, timing, and vocal timbre, even if their overall impressions are subjective. Work by Ghisingh et al. (2017) [1] analyzed Indian classical singing by isolating the vocal track from background music and examining acoustic production features like pitch and root-mean-square energy. Other researchers have built systems to grade singing on metrics including intonation correctness, rhythm consistency, and vibrato usage. For instance, Liu and Wallmark (2024) [2] trained machine learning models on annotated singer characteristics to classify timbre and technique attributes in traditional opera singing. Meanwhile, recent surveys by Hashem et al. (2023) [3] document speech emotion recognition systems that infer emotions from voice signals via deep learning. These advances illustrate that modern AI and machine learning (ML) can decipher subtle information from vocal audio, whether it be emotional tone or singing skill.

While classical ML approaches for voice grading are promising, nascent technologies like quantum computing open new frontiers for audio analysis. Quantum computing’s ability to represent and entangle high-dimensional data offers a novel way to capture the complex interdependencies of musical features. Kashani et al. (2022) [4] implemented a note detection algorithm based on the Quantum Fourier Transform (QFT). Miranda et al. (2021) [5] envisioned quantum computing’s impact on music technology and introduced frameworks for quantum music intelligence. Gündüz (2023) [6] used information-theoretic metrics like Shannon entropy to quantify musical complexity. These developments set the stage for a hybrid approach to the long-standing challenge of vocal performance evaluation.

QuantumMelody is proposed as a solution that combines robust audio feature analysis with quantum computing to achieve consistent and insightful vocal grading. First, the algorithm extracts a feature vector from raw singing audio, encompassing both traditional and innovative descriptors of vocal quality. These features capture pitch and intonation accuracy, frequency stability (jitter) and amplitude stability (shimmer), LUFS energy (loudness and dynamics), and timbral characteristics such as MFCCs and formant frequencies. Expressive elements like vibrato extent and rate are also included.

II Materials and Methods

II-A Dataset and Preprocessing

We collect $20\text{\,}\mathrm{s}$ vocal recordings in seven Hindustani ragas, with 42 labeled samples each for ratings of 2, 3, 4 and 5 across the seven ragas, for a total of 168 samples. All audio is converted to mono and resampled to $22\,050\text{\,}\mathrm{H}\mathrm{z}$ using Librosa’s high-quality resampler. We then apply denoising and drone removal by attenuating low-frequency harmonics corresponding to a tanpura drone via a narrow band-stop filter around the drone frequency. This is followed by harmonic–percussive separation to extract the pure vocal track. After filtering, we use a short-time Fourier transform (STFT) [1] to verify that the drone, percussion, and any low-frequency noise are removed while preserving the singer’s voice.

II-B Feature Extraction

We extract a suite of time-domain and frequency-domain features for each recording as follows.

Pitch deviation (cents)

Let the instantaneous fundamental frequency be $F_{0}[n]$ and the tonic be $F_{\mathrm{ref}}$. The per-frame pitch deviation in cents is

[TABLE]

We compute the average absolute pitch deviation $\tfrac{1}{N}\sum_{n=1}^{N}|\Delta[n]|$, which quantifies pitch stability.

Jitter

With successive pitch periods $T_{i}=1/F_{0}[i]$, the local jitter is

[TABLE]

reported as a percentage.

Shimmer

With glottal pulse peak amplitudes $A_{i}$, the local shimmer is

[TABLE]

Loudness (LUFS) and RMS energy

ITU-R BS.1770 loudness is approximated by

[TABLE]

where $w[n]$ is the K-weighted waveform. We also compute

[TABLE]

Tone-to-Noise Ratio (TNR)

Using Parselmouth, we estimate

[TABLE]

MFCCs

From log-Mel energies $S_{m}$, the $k$th MFCC is

[TABLE]

Zero-Crossing Rate (ZCR)

The short-term zero-crossing rate per frame is

[TABLE]

Spectral centroid

[TABLE]

Spectral bandwidth

[TABLE]

Spectral flatness

[TABLE]

where $P_{k}=|X(f_{k})|^{2}$.

Formants F1–F3

Using Burg LPC we estimate the first three formants per voiced frame and take their means (Hz).

Vibrato extent and rate

For the pitch contour $F_{0}[n]$, the vibrato extent (in cents) is

[TABLE]

and the rate is the dominant modulation frequency of $F_{0}[n]$ (Hz).

II-C Angle Scaling

After computing raw features, we scale them to angles in $[0,2\pi]$ to map to the Bloch sphere and construct the quantum circuit. Representative mappings include:

[TABLE]

Here, $T_{\max}$ and $T_{\min}$ are the empirical bounds for TNR (dB); analogous symbols are used for other features. Unless noted, scaling parameters ($d_{0},k,a,b,S_{0},r,\sigma_{\max},M_{\max}$) are fixed from the training set using robust bounds (5th–95th percentile or empirical min/max as indicated) and remain constant during evaluation.

II-D Quantum Circuit Architecture

We construct a 9-qubit circuit in Qiskit. All qubits are initialized with a Hadamard layer $H^{\otimes 9}$, then receive rotations based on grouped features: qubits 0–2 receive $R_{x}(\theta_{i})$ (pitch-related angles), 3–5 receive $R_{y}(\theta_{i})$ (dynamics), and 6–8 receive $R_{z}(\theta_{i})$ (timbre). We apply intra-group CNOTs (0$\rightarrow$1, 1$\rightarrow$2; 3$\rightarrow$4, 4$\rightarrow$5; 6$\rightarrow$7, 7$\rightarrow$8) and cross-group CNOTs (2$\rightarrow$3, 1$\rightarrow$4, 0$\rightarrow$6, 5$\rightarrow$7), then measure all qubits with $8192$ shots on the Qiskit simulator to obtain measurement probabilities.

II-E Hybrid Neural Network

We combine an Audio Spectrogram Transformer (AST) [7] fine-tuned on Mel-spectrograms with parallel MLPs over classical features and quantum-derived features. The concatenated embeddings are fed to a fully-connected head that predicts a categorical grade (2–5).

II-F Comparison Metrics and Environment

For two recordings with quantum probability distributions $P$ and $Q$, the Jensen–Shannon divergence is

[TABLE]

where $D_{KL}(\cdot\|\cdot)$ denotes the Kullback–Leibler divergence. We also compute the Euclidean distance between scaled-angle vectors $\boldsymbol{\theta}^{(1)}$ and $\boldsymbol{\theta}^{(2)}$:

[TABLE]

All code is written in Python 3.12 using Librosa, Parselmouth, Qiskit, PennyLane and PyTorch on macOS.

II-G Ethics and Reproducibility

All participants provided informed consent for recording and research use of their vocal audio. Data are anonymized; no identifying metadata are released.

Audio is mono at $22.05\text{\,}\mathrm{kHz}$. Feature set: $F_{0}$ deviation (cents), jitter, shimmer, LUFS/RMS, TNR, MFCCs (first 13, $\mu/\sigma$), ZCR, spectral centroid/bandwidth/flatness, formants, and vibrato extent/rate. The quantum circuit configuration and training code are available on request; a companion repository with scripts and hyperparameters will be posted after this preprint is announced.

III Results and Discussion

The hybrid classical–quantum framework processed each vocal recording in approximately one minute on average (Qiskit simulator). We do not claim hardware speedups; real-time performance on quantum hardware is outside our scope. Fig. 1 shows the classical metrics captured for each recording and used as inputs to the combined model.

Quantum measurement distributions were calculated for each circuit using $8192$ shots (Fig. 2). Most recordings did not display a single spike but showed a clear shape, indicating intertwined features.

III-A Improvement over Classical Methods

The baseline included classical features only (pitch dev., jitter/shimmer, MFCC stats, TNR, LUFS, ZCR, formants) with an MLP; the hybrid model adds AST embeddings + quantum measurement probabilities. We used an 80/20 stratified split with raga/label balance and no speaker leakage. We report agreement, defined as the percentage of samples whose predicted label exactly matches the expert-provided grade. We cast 2–5 grading as a four-class classification task and report agreement with expert graders. Compared to the classical-only baseline, the hybrid model achieved a +$12.86$-point absolute improvement in agreement with expert graders, reaching $74.29\text{\,}\mathrm{\char 37\relax}$ versus $61.43\text{\,}\mathrm{\char 37\relax}$ for the classical system (Fig. 3). Given n=168, estimates have non-trivial variance; we report point estimates here.

III-B Student vs. Master Comparison

Divergence measures—Jensen–Shannon and Euclidean—computed on quantum-encoded feature vectors qualitatively aligned with perceptual discrepancies between student and teacher recordings. Analysis showed pitch deviations of 15–25 cents from master tracks (ideal $<$10 cents), LUFS differences up to $3\text{\,}\mathrm{d}\mathrm{B}$, and TNR values of $12\text{\,}\mathrm{d}\mathrm{B}18\text{\,}\mathrm{d}\mathrm{B}$ for students vs. $>$20\text{,}\mathrm{d}\mathrm{B}$$ for teachers (Fig. 5). Training curves and confusion matrices for the AST model are in Fig. 4.

IV Conclusion

We presented QuantumMelody, a hybrid method that encodes grouped vocal features in a compact quantum circuit and fuses the circuit’s measurement probabilities with spectrogram-transformer embeddings to estimate a 2–5 grade and surface technique-level feedback. The circuit uses nine qubits with $R_{x}$, $R_{y}$, and $R_{z}$ encodings aligned respectively with pitch stability, dynamics, and timbre, with intra- and inter-group entanglement to model cross-domain interactions.

On $168$ labeled $20\text{\,}\mathrm{s}$ excerpts, the hybrid model attains $74.29\text{\,}\mathrm{\char 37\relax}$ agreement with expert graders, a +$12.86$-point improvement over a classical-features baseline. All quantum results are produced on a laptop-class Qiskit simulator; we do not claim hardware speedups, and behavior on current NISQ devices may differ. Given the modest, single-domain dataset, these findings should be interpreted as a feasibility demonstration within applied audio signal processing.

Overall, the approach provides objective, interpretable indicators for vocal technique without relying on quantum hardware performance claims. This work is most appropriately read within applied audio signal processing.