ViTO: Vision Transformer-Operator

TL;DR

ViTO introduces a novel approach combining vision transformers with operator learning to efficiently solve complex inverse PDE problems, achieving high accuracy with fewer parameters and faster performance.

Contribution

The paper presents ViTO, a new architecture that integrates vision transformers with U-Net for inverse PDE problems, outperforming existing benchmarks in accuracy and efficiency.

Findings

Achieves comparable or better accuracy than leading operator networks.

Uses less than 10% of parameters of competitors, enabling faster inference.

Successfully solves inverse PDE problems including wave, Navier-Stokes, and Darcy equations.

Abstract

We combine vision transformers with operator learning to solve diverse inverse problems described by partial differential equations (PDEs). Our approach, named ViTO, combines a U-Net based architecture with a vision transformer. We apply ViTO to solve inverse PDE problems of increasing complexity, namely for the wave equation, the Navier-Stokes equations and the Darcy equation. We focus on the more challenging case of super-resolution, where the input dataset for the inverse problem is at a significantly coarser resolution than the output. The results we obtain are comparable or exceed the leading operator network benchmarks in terms of accuracy. Furthermore, ViTO`s architecture has a small number of trainable parameters (less than 10% of the leading competitor), resulting in a performance speed-up of over 5x when averaged over the various test cases.