Learning Feature Interactions with Lorentzian Factorization Machine

TL;DR

This paper introduces LorentzFM, a hyperbolic space-based model for feature interaction learning that reduces parameters and computational costs while maintaining or improving prediction accuracy in recommendation systems.

Contribution

LorentzFM leverages hyperbolic geometry to efficiently learn feature interactions with fewer parameters, outperforming some deep learning models in recommendation tasks.

Findings

LorentzFM reduces model parameters by 20-80%.

LorentzFM achieves comparable or better accuracy than deep learning methods.

The hyperbolic embedding improves feature interaction modeling.

Abstract

Learning representations for feature interactions to model user behaviors is critical for recommendation system and click-trough rate (CTR) predictions. Recent advances in this area are empowered by deep learning methods which could learn sophisticated feature interactions and achieve the state-of-the-art result in an end-to-end manner. These approaches require large number of training parameters integrated with the low-level representations, and thus are memory and computational inefficient. In this paper, we propose a new model named "LorentzFM" that can learn feature interactions embedded in a hyperbolic space in which the violation of triangle inequality for Lorentz distances is available. To this end, the learned representation is benefited by the peculiar geometric properties of hyperbolic triangles, and result in a significant reduction in the number of parameters (20\% to 80\%) because all the top deep learning layers are not required. With such a lightweight architecture, LorentzFM achieves comparable and even materially better results than the deep learning methods such as DeepFM, xDeepFM and Deep \& Cross in both recommendation and CTR prediction tasks.