Accelerating Automatic Differentiation of Direct Form Digital Filters

TL;DR

This paper presents a novel approach for automatic differentiation of direct form digital filters, enabling faster GPU implementations and analytical gradients that outperform traditional methods in speed.

Contribution

It introduces a unified formulation for backpropagation through direct form filters, including initial condition gradients, with efficient GPU implementations.

Findings

C++/CUDA implementation achieves 1000x speedup over naive Python

Exact time-domain filtering outperforms frequency-domain methods for low-order filters

Open-source code available at GitHub

Abstract

We introduce a general formulation for automatic differentiation through direct form filters, yielding a closed-form backpropagation that includes initial condition gradients. The result is a single expression that can represent both the filter and its gradients computation while supporting parallelism. C++/CUDA implementations in PyTorch achieve at least 1000x speedup over naive Python implementations and consistently run fastest on the GPU. For the low-order filters commonly used in practice, exact time-domain filtering with analytical gradients outperforms the frequency-domain method in terms of speed. The source code is available at https://github.com/yoyolicoris/philtorch.