Generalized vec trick for fast learning of pairwise kernel models

TL;DR

This paper reviews pairwise kernels and introduces a generalized vec trick algorithm that significantly accelerates training, enabling scalable pairwise kernel learning for large biological datasets.

Contribution

It demonstrates how all reviewed kernels can be expressed as sums of Kronecker products, allowing the generalized vec trick to speed up computation and scale to larger datasets.

Findings

The generalized vec trick reduces training time from quadratic to linear in data size.

All reviewed kernels can be expressed as sums of Kronecker products for efficient computation.

The approach enables scaling pairwise kernel methods to larger biological datasets.

Abstract

Pairwise learning corresponds to the supervised learning setting where the goal is to make predictions for pairs of objects. Prominent applications include predicting drug-target or protein-protein interactions, or customer-product preferences. In this work, we present a comprehensive review of pairwise kernels, that have been proposed for incorporating prior knowledge about the relationship between the objects. Specifically, we consider the standard, symmetric and anti-symmetric Kronecker product kernels, metric-learning, Cartesian, ranking, as well as linear, polynomial and Gaussian kernels. Recently, a O(nm + nq) time generalized vec trick algorithm, where n, m, and q denote the number of pairs, drugs and targets, was introduced for training kernel methods with the Kronecker product kernel. This was a significant improvement over previous O(n^2) training methods, since in most real-world applications m,q << n. In this work we show how all the reviewed kernels can be expressed as sums of Kronecker products, allowing the use of generalized vec trick for speeding up their computation. In the experiments, we demonstrate how the introduced approach allows scaling pairwise kernels to much larger data sets than previously feasible, and provide an extensive comparison of the kernels on a number of biological interaction prediction tasks.