Cubic Spline Smoothing Compensation for Irregularly Sampled Sequences

TL;DR

This paper introduces a cubic spline smoothing compensation module for ODE-RNNs to address interpolation discontinuities in irregularly sampled sequences, demonstrating improved performance and theoretical guarantees.

Contribution

It proposes a novel cubic spline smoothing compensation module for ODE-RNNs, with an analytical solution and theoretical error bounds, enhancing sequence modeling.

Findings

Outperforms ODE-RNN and cubic spline interpolation in experiments

Provides an analytical solution for the smoothing module

Establishes theoretical bounds on interpolation error

Abstract

The marriage of recurrent neural networks and neural ordinary differential networks (ODE-RNN) is effective in modeling irregularly-observed sequences. While ODE produces the smooth hidden states between observation intervals, the RNN will trigger a hidden state jump when a new observation arrives, thus cause the interpolation discontinuity problem. To address this issue, we propose the cubic spline smoothing compensation, which is a stand-alone module upon either the output or the hidden state of ODE-RNN and can be trained end-to-end. We derive its analytical solution and provide its theoretical interpolation error bound. Extensive experiments indicate its merits over both ODE-RNN and cubic spline interpolation.