Neural Approximation and Its Applications

TL;DR

This paper introduces Neural Approximation (NeuApprox), a novel method using neural basis functions for multivariate function approximation, enhancing flexibility and accuracy over traditional basis functions, with theoretical guarantees and diverse experimental validation.

Contribution

The paper proposes neural basis functions and the NeuApprox framework, enabling improved approximation and data adaptation for multivariate functions, surpassing classic hand-crafted basis methods.

Findings

NeuApprox achieves high approximation accuracy on diverse datasets.

NeuApprox demonstrates superior data adaptation and fine-tuning capabilities.

Theoretical proof confirms NeuApprox's ability to approximate any continuous multivariate function.

Abstract

Multivariate function approximation is a fundamental problem in machine learning. Classic multivariate function approximations rely on hand-crafted basis functions (e.g., polynomial basis and Fourier basis), which limits their approximation ability and data adaptation ability, resulting in unsatisfactory performance. To address these challenges, we introduce the neural basis function by leveraging an untrained neural network as the basis function. Equipped with the proposed neural basis function, we suggest the neural approximation (NeuApprox) paradigm for multivariate function approximation. Specifically, the underlying multivariate function behind the multi-dimensional data is decomposed into a sum of block terms. The clear physically-interpreted block term is the product of expressive neural basis functions and their corresponding learnable coefficients, which allows us to faithfully capture distinct components of the underlying data and also flexibly adapt to new data by readily fine-tuning the neural basis functions. Attributed to the elaborately designed block terms, the suggested NeuApprox enjoys strong approximation ability and flexible data adaptation ability over the hand-crafted basis function-based methods. We also theoretically prove that NeuApprox can approximate any multivariate continuous function to arbitrary accuracy. Extensive experiments on diverse multi-dimensional datasets (including multispectral images, light field data, videos, traffic data, and point cloud data) demonstrate the promising performance of NeuApprox in terms of both approximation capability and adaptability.