Learning Functional Transduction

TL;DR

This paper introduces a hybrid meta-learning approach for transductive regression that efficiently models complex functions with minimal data, leveraging RKBS theory to enable rapid inference in physical system modeling.

Contribution

It presents a novel method to meta-learn transductive regression principles using gradient descent, creating neural approximators that can instantly adapt to new data.

Findings

Effective in modeling physical systems with limited data

Reduces training computational cost significantly

Capable of capturing infinite functional relationships

Abstract

Research in machine learning has polarized into two general approaches for regression tasks: Transductive methods construct estimates directly from available data but are usually problem unspecific. Inductive methods can be much more specific but generally require compute-intensive solution searches. In this work, we propose a hybrid approach and show that transductive regression principles can be meta-learned through gradient descent to form efficient in-context neural approximators by leveraging the theory of vector-valued Reproducing Kernel Banach Spaces (RKBS). We apply this approach to function spaces defined over finite and infinite-dimensional spaces (function-valued operators) and show that once trained, the Transducer can almost instantaneously capture an infinity of functional relationships given a few pairs of input and output examples and return new image estimates. We demonstrate the benefit of our meta-learned transductive approach to model complex physical systems influenced by varying external factors with little data at a fraction of the usual deep learning training computational cost for partial differential equations and climate modeling applications.