DON-LSTM: Multi-Resolution Learning with DeepONets and Long Short-Term Memory Neural Networks

TL;DR

This paper introduces DON-LSTM, a novel neural network architecture combining DeepONets and LSTMs, designed to effectively learn from multi-resolution data and capture long-term dependencies in complex systems.

Contribution

The paper presents DON-LSTM, a new architecture that enhances DeepONets with LSTM capabilities for improved long-sequence learning from multi-resolution data.

Findings

Achieves lower generalization error on long-time-evolution tasks.

Requires fewer high-resolution samples than traditional DeepONets.

Effectively captures temporal dependencies in nonlinear systems.

Abstract

Deep operator networks (DeepONets, DONs) offer a distinct advantage over traditional neural networks in their ability to be trained on multi-resolution data. This property becomes especially relevant in real-world scenarios where high-resolution measurements are difficult to obtain, while low-resolution data is more readily available. Nevertheless, DeepONets alone often struggle to capture and maintain dependencies over long sequences compared to other state-of-the-art algorithms. We propose a novel architecture, named DON-LSTM, which extends the DeepONet with a long short-term memory network (LSTM). Combining these two architectures, we equip the network with explicit mechanisms to leverage multi-resolution data, as well as capture temporal dependencies in long sequences. We test our method on long-time-evolution modeling of multiple non-linear systems and show that the proposed multi-resolution DON-LSTM achieves significantly lower generalization error and requires fewer high-resolution samples compared to its vanilla counterparts.