RTNeural: Fast Neural Inferencing for Real-Time Systems

TL;DR

RTNeural is a C++ library optimized for real-time neural inferencing, emphasizing speed, flexibility, and small size, suitable for systems with strict timing constraints.

Contribution

It introduces a new neural inferencing library tailored for real-time applications, with detailed design, use-cases, and performance comparisons.

Findings

RTNeural achieves faster inference times than comparable libraries.

The library is highly flexible and compact, suitable for embedded systems.

Performance benchmarks demonstrate its suitability for real-time constraints.

Abstract

RTNeural is a neural inferencing library written in C++. RTNeural is designed to be used in systems with hard real-time constraints, with additional emphasis on speed, flexibility, size, and convenience. The motivation and design of the library are described, as well as real-world use-cases, and performance comparisons with other neural inferencing libraries.