FiBiNet++: Reducing Model Size by Low Rank Feature Interaction Layer for CTR Prediction

TL;DR

FiBiNet++ introduces a low-rank feature interaction layer to significantly reduce model size and enhance performance in CTR prediction, outperforming existing models on multiple datasets.

Contribution

The paper proposes FiBiNet++, a redesigned model with a low-rank layer that compresses size and improves accuracy over FiBiNet and other CTR models.

Findings

Reduces FiBiNet's parameters by 12x to 16x.

Achieves better CTR prediction performance.

Outperforms state-of-the-art models on three datasets.

Abstract

Click-Through Rate (CTR) estimation has become one of the most fundamental tasks in many real-world applications and various deep models have been proposed. Some research has proved that FiBiNet is one of the best performance models and outperforms all other models on Avazu dataset. However, the large model size of FiBiNet hinders its wider application. In this paper, we propose a novel FiBiNet++ model to redesign FiBiNet's model structure, which greatly reduces model size while further improves its performance. One of the primary techniques involves our proposed "Low Rank Layer" focused on feature interaction, which serves as a crucial driver of achieving a superior compression ratio for models. Extensive experiments on three public datasets show that FiBiNet++ effectively reduces non-embedding model parameters of FiBiNet by 12x to 16x on three datasets. On the other hand, FiBiNet++ leads to significant performance improvements compared to state-of-the-art CTR methods, including FiBiNet.