Polyharmonic Spline Packages: Composition, Efficient Procedures for Computation and Differentiation

TL;DR

This paper introduces a scalable cascade architecture of polyharmonic spline packages for machine learning regression, providing efficient computation and differentiation methods, and addressing high-dimensional challenges with theoretical justification.

Contribution

It proposes a novel cascade architecture of polyharmonic splines that improves scalability and computational efficiency for high-dimensional regression problems.

Findings

Efficient matrix procedures enable fast forward computation.

End-to-end differentiation is achievable through the cascade.

The approach is theoretically justified for low intrinsic dimensionality.

Abstract

In a previous paper it was shown that a machine learning regression problem can be solved within the framework of random function theory, with the optimal kernel analytically derived from symmetry and indifference principles and coinciding with a polyharmonic spline. However, a direct application of that solution is limited by O(N^3) computational cost and by a breakdown of the original theoretical assumptions when the input space has excessive dimensionality. This paper proposes a cascade architecture built from packages of polyharmonic splines that simultaneously addresses scalability and is theoretically justified for problems with unknown intrinsic low dimensionality. Efficient matrix procedures are presented for forward computation and end-to-end differentiation through the cascade.