Space-efficient Feature Maps for String Alignment Kernels

TL;DR

This paper introduces a novel, space-efficient approximation method for string alignment kernels, enabling large-scale string classification with improved scalability, accuracy, and computational efficiency.

Contribution

It presents the first space-efficient feature maps for string alignment kernels, reducing memory usage from quadratic to linear while maintaining theoretical guarantees.

Findings

SFMEDM outperforms existing methods in prediction accuracy

The method demonstrates superior scalability to large datasets

It significantly reduces memory consumption in large-scale string analysis

Abstract

String kernels are attractive data analysis tools for analyzing string data. Among them, alignment kernels are known for their high prediction accuracies in string classifications when tested in combination with SVM in various applications. However, alignment kernels have a crucial drawback in that they scale poorly due to their quadratic computation complexity in the number of input strings, which limits large-scale applications in practice. We address this need by presenting the first approximation for string alignment kernels, which we call space-efficient feature maps for edit distance with moves (SFMEDM), by leveraging a metric embedding named edit sensitive parsing (ESP) and feature maps (FMs) of random Fourier features (RFFs) for large-scale string analyses. The original FMs for RFFs consume a huge amount of memory proportional to the dimension d of input vectors and the dimension D of output vectors, which prohibits its large-scale applications. We present novel space-efficient feature maps (SFMs) of RFFs for a space reduction from O(dD) of the original FMs to O(d) of SFMs with a theoretical guarantee with respect to concentration bounds. We experimentally test SFMEDM on its ability to learn SVM for large-scale string classifications with various massive string data, and we demonstrate the superior performance of SFMEDM with respect to prediction accuracy, scalability and computation efficiency.