SaFARi: State-Space Models for Frame-Agnostic Representation

TL;DR

SaFARi introduces a flexible framework for state-space models that can incorporate any basis or frame, extending beyond polynomial bases and enabling diverse applications in long-range data modeling.

Contribution

It presents a generalized method for constructing SSMs with arbitrary bases, broadening the scope beyond traditional polynomial-based models.

Findings

Enables SSM construction with any frame or basis.

Includes the HiPPO approach as a special case.

Allows for an infinite diversity of SSM architectures.

Abstract

State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data. However, to date, only a few polynomial bases have been explored for this purpose, and the state-of-the-art implementations were built upon the best of a few limited options. In this paper, we present a generalized method for building an SSM with any frame or basis, rather than being restricted to polynomials. This framework encompasses the approach known as HiPPO, but also permits an infinite diversity of other possible "species" within the SSM architecture. We dub this approach SaFARi: SSMs for Frame-Agnostic Representation.