PeakNetFP: Peak-based Neural Audio Fingerprinting Robust to Extreme Time Stretching

TL;DR

PeakNetFP is a novel neural audio fingerprinting system that leverages spectral peaks and hierarchical feature extraction to achieve high accuracy and efficiency, especially under extreme time stretching conditions.

Contribution

It introduces PeakNetFP, combining peak-based spectral features with neural network techniques, achieving robustness and efficiency in audio fingerprinting tasks.

Findings

Over 90% Top-1 hit rate for 50%-200% time stretching

100 times fewer parameters than NeuralFP

Uses 11 times less input data

Abstract

This work introduces PeakNetFP, the first neural audio fingerprinting (AFP) system designed specifically around spectral peaks. This novel system is designed to leverage the sparse spectral coordinates typically computed by traditional peak-based AFP methods. PeakNetFP performs hierarchical point feature extraction techniques similar to the computer vision model PointNet++, and is trained using contrastive learning like in the state-of-the-art deep learning AFP, NeuralFP. This combination allows PeakNetFP to outperform conventional AFP systems and achieves comparable performance to NeuralFP when handling challenging time-stretched audio data. In extensive evaluation, PeakNetFP maintains a Top-1 hit rate of over 90% for stretching factors ranging from 50% to 200%. Moreover, PeakNetFP offers significant efficiency advantages: compared to NeuralFP, it has 100 times fewer parameters and uses 11 times smaller input data. These features make PeakNetFP a lightweight and efficient solution for AFP tasks where time stretching is involved. Overall, this system represents a promising direction for future AFP technologies, as it successfully merges the lightweight nature of peak-based AFP with the adaptability and pattern recognition capabilities of neural network-based approaches, paving the way for more scalable and efficient solutions in the field.