Parameter-Efficient Neural CDEs via Implicit Function Jacobians

TL;DR

This paper introduces a parameter-efficient approach to Neural Controlled Differential Equations (Neural CDEs), reducing complexity while maintaining their ability to analyze temporal sequences, and drawing an analogy to continuous RNNs.

Contribution

It proposes a novel, parameter-efficient variant of Neural CDEs that simplifies the model and offers a logical analogy to continuous RNNs.

Findings

Significantly fewer parameters needed for Neural CDEs.

Maintains effectiveness in analyzing temporal sequences.

Establishes a clear analogy to continuous RNNs.

Abstract

Neural Controlled Differential Equations (Neural CDEs, NCDEs) are a unique branch of methods, specifically tailored for analysing temporal sequences. However, they come with drawbacks, the main one being the number of parameters, required for the method's operation. In this paper, we propose an alternative, parameter-efficient look at Neural CDEs. It requires much fewer parameters, while also presenting a very logical analogy as the "Continuous RNN", which the Neural CDEs aspire to.