A resource-efficient model for deep kernel learning

TL;DR

This paper introduces a novel model-level decomposition approach for deep kernel learning that aims to reduce computational complexity while maintaining accuracy, addressing the curse of dimensionality.

Contribution

It presents a new mathematical decomposition method at the model level, combining operator and network decomposition for improved scalability.

Findings

Feasibility of the proposed approach demonstrated.

Maintains accuracy with reduced computational complexity.

Addresses challenges of high-dimensional data.

Abstract

According to the Hughes phenomenon, the major challenges encountered in computations with learning models comes from the scale of complexity, e.g. the so-called curse of dimensionality. There are various approaches for accelerate learning computations with minimal loss of accuracy. These approaches range from model-level to implementation-level approaches. To the best of our knowledge, the first one is rarely used in its basic form. Perhaps, this is due to theoretical understanding of mathematical insights of model decomposition approaches, and thus the ability of developing mathematical improvements has lagged behind. We describe a model-level decomposition approach that combines both the decomposition of the operators and the decomposition of the network. We perform a feasibility analysis on the resulting algorithm, both in terms of its accuracy and scalability.