ChladniSonify: A Visual-Acoustic Mapping Method for Chladni Patterns in New Media Art Creation

TL;DR

ChladniSonify is a real-time system that maps Chladni patterns to sound, enabling interactive audio-visual art creation with high accuracy and low latency.

Contribution

It introduces a novel real-time visual-acoustic mapping method using a lightweight CNN and finite element simulation, improving interactivity and precision in Chladni pattern sonification.

Findings

Classification accuracy of 99.33% on test set

Inference latency of 7.03 ms for pattern recognition

End-to-end latency under 50 ms for real-time interaction

Abstract

In new media art creation, the mapping between vision and hearing is often subjective. As a classic carrier of sound visualization, Chladni patterns have great potential in building audio-visual mapping mechanisms. However, existing tools face pain points: high technical barriers for simulation, offline computing failing real-time interaction, and uncontrollable mapping rules in general sonification tools. To address these, this paper proposes ChladniSonify, a real-time visual-acoustic mapping method for Chladni patterns. Based on Kirchhoff-Love plate theory, we build a paired dataset via numerical programming and calibrate it using ANSYS finite element simulation. Focusing on the slender nodal lines of Chladni patterns, we adopt a lightweight CNN with CBAM to achieve high-precision, low-latency pattern classification. Finally, we build an end-to-end system in Python and Max/MSP, mapping recognized patterns to corresponding sine wave frequencies. Results show the system has excellent usability: the classification module achieves 99.33% accuracy on the test set with 7.03 ms inference latency; the mapped frequency matches the theoretical value with zero deviation; the average end-to-end latency is under 50 ms, meeting real-time interactive needs. This work provides a reproducible engineering prototype for Chladni audio-visual art creation.