Sample-efficient Surrogate Model for Frequency Response of Linear PDEs using Self-Attentive Complex Polynomials

TL;DR

This paper introduces a novel, sample-efficient surrogate model for linear PDEs, specifically applied to antenna design, leveraging a new parametric framework and attention-based neural networks to improve prediction accuracy and design success rates.

Contribution

It proposes the CZP framework for linear PDEs, enabling direct prediction of physical quantities in the Fourier domain, and combines it with a neural network architecture for improved antenna design.

Findings

Outperforms baselines by 10-25% in test loss.

Achieves 33% higher success in antenna design verification.

Demonstrates sample efficiency in surrogate modeling for linear PDEs.

Abstract

Linear Partial Differential Equations (PDEs) govern the spatial-temporal dynamics of physical systems that are essential to building modern technology. When working with linear PDEs, designing a physical system for a specific outcome is difficult and costly due to slow and expensive explicit simulation of PDEs and the highly nonlinear relationship between a system's configuration and its behavior. In this work, we prove a parametric form that certain physical quantities in the Fourier domain must obey in linear PDEs, named the CZP (Constant-Zeros-Poles) framework. Applying CZP to antenna design, an industrial application using linear PDEs (i.e., Maxwell's equations), we derive a sample-efficient parametric surrogate model that directly predicts its scattering coefficients without explicit numerical PDE simulation. Combined with a novel image-based antenna representation and an attention-based neural network architecture, CZP outperforms baselines by 10% to 25% in terms of test loss and also is able to find 2D antenna designs verifiable by commercial software with $33\%$ greater success than baselines, when coupled with sequential search techniques like reinforcement learning.