ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications

TL;DR

ANIRA is a cross-platform library designed to enable real-time neural network inference for audio applications, supporting multiple frameworks and incorporating latency management and benchmarking.

Contribution

It introduces anira, a novel library that decouples inference from audio callbacks and supports various backends, improving real-time performance in audio neural network inference.

Findings

ONNX Runtime has the lowest runtimes for stateless models

LibTorch performs fastest for stateful models

Initial inferences can be longer, especially with real-time violations

Abstract

Numerous tools for neural network inference are currently available, yet many do not meet the requirements of real-time audio applications. In response, we introduce anira, an efficient cross-platform library. To ensure compatibility with a broad range of neural network architectures and frameworks, anira supports ONNX Runtime, LibTorch, and TensorFlow Lite as backends. Each inference engine exhibits real-time violations, which anira mitigates by decoupling the inference from the audio callback to a static thread pool. The library incorporates built-in latency management and extensive benchmarking capabilities, both crucial to ensure a continuous signal flow. Three different neural network architectures for audio effect emulation are then subjected to benchmarking across various configurations. Statistical modeling is employed to identify the influence of various factors on performance. The findings indicate that for stateless models, ONNX Runtime exhibits the lowest runtimes. For stateful models, LibTorch demonstrates the fastest performance. Our results also indicate that for certain model-engine combinations, the initial inferences take longer, particularly when these inferences exhibit a higher incidence of real-time violations.