Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch

TL;DR

Continual Inference introduces a Python library for efficient online and batch inference with Deep Neural Networks in PyTorch, facilitating practical implementation of Continual Inference Networks (CINs).

Contribution

The paper presents a new Python library, Continual Inference, that simplifies the implementation of CINs for efficient neural network inference.

Findings

Provides a comprehensive guide and code examples for CIN implementation

Enables efficient online and batch inference in deep learning workflows

Available as an open-source package on PyPI and GitHub

Abstract

We present Continual Inference, a Python library for implementing Continual Inference Networks (CINs) in PyTorch, a class of Neural Networks designed specifically for efficient inference in both online and batch processing scenarios. We offer a comprehensive introduction and guide to CINs and their implementation in practice, and provide best-practices and code examples for composing complex modules for modern Deep Learning. Continual Inference is readily downloadable via the Python Package Index and at \url{www.github.com/lukashedegaard/continual-inference}.