Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks

TL;DR

This paper provides a theoretical analysis of the approximation capabilities of linear temporal convolutional networks, including new rate estimates and inverse theorems that characterize their efficiency in modeling sequential data.

Contribution

It introduces a refined complexity measure for better approximation rate estimates and presents a novel inverse approximation theorem for temporal convolutional architectures.

Findings

Improved approximation rate estimate with a new complexity measure

First inverse approximation theorem for temporal convolutional networks

Comprehensive characterization of sequential relationships captured by these networks

Abstract

We present a theoretical analysis of the approximation properties of convolutional architectures when applied to the modeling of temporal sequences. Specifically, we prove an approximation rate estimate (Jackson-type result) and an inverse approximation theorem (Bernstein-type result), which together provide a comprehensive characterization of the types of sequential relationships that can be efficiently captured by a temporal convolutional architecture. The rate estimate improves upon a previous result via the introduction of a refined complexity measure, whereas the inverse approximation theorem is new.