tensorFM: Low-Rank Approximations of Cross-Order Feature Interactions

TL;DR

TensorFM is a novel model that efficiently captures high-order feature interactions in categorical data using low-rank tensor approximations, offering competitive accuracy and low latency for real-time applications.

Contribution

Introduces tensorFM, a new low-rank tensor-based model that generalizes field-weighted factorization machines for high-order feature interaction modeling.

Findings

TensorFM achieves competitive performance with state-of-the-art methods.

The model offers low latency suitable for online applications.

Empirical results validate the effectiveness of tensorFM.

Abstract

We address prediction problems on tabular categorical data, where each instance is defined by multiple categorical attributes, each taking values from a finite set. These attributes are often referred to as fields, and their categorical values as features. Such problems frequently arise in practical applications, including click-through rate prediction and social sciences. We introduce and analyze {tensorFM}, a new model that efficiently captures high-order interactions between attributes via a low-rank tensor approximation representing the strength of these interactions. Our model generalizes field-weighted factorization machines. Empirically, tensorFM demonstrates competitive performance with state-of-the-art methods. Additionally, its low latency makes it well-suited for time-sensitive applications, such as online advertising.