Regression with Linear Factored Functions

TL;DR

This paper introduces a regression algorithm for linear factored functions (LFF) that leverages their structural properties to efficiently compute integrals and products, addressing the curse of dimensionality in high-dimensional applications.

Contribution

The paper presents a novel regression algorithm for LFF that enables analytical integral and product computations, improving efficiency in high-dimensional problems.

Findings

Performs competitively with Gaussian processes on benchmarks.

Learned LFF functions are very compact, with 4-9 basis functions.

Exploits structural properties to break the curse of dimensionality.

Abstract

Many applications that use empirically estimated functions face a curse of dimensionality, because the integrals over most function classes must be approximated by sampling. This paper introduces a novel regression-algorithm that learns linear factored functions (LFF). This class of functions has structural properties that allow to analytically solve certain integrals and to calculate point-wise products. Applications like belief propagation and reinforcement learning can exploit these properties to break the curse and speed up computation. We derive a regularized greedy optimization scheme, that learns factored basis functions during training. The novel regression algorithm performs competitively to Gaussian processes on benchmark tasks, and the learned LFF functions are with 4-9 factored basis functions on average very compact.