EMOFM: Ensemble MLP mOdel with Feature-based Mixers for Click-Through Rate Prediction

TL;DR

This paper introduces EMOFM, a novel ensemble MLP model with feature-based mixers designed for click-through rate prediction, effectively fusing features across fields and types to improve prediction accuracy on large, privacy-preserving datasets.

Contribution

The paper proposes a new plug-in mixer architecture for field and type-wise feature fusion within an ensemble MLP framework, tailored for CTR prediction tasks.

Findings

EMOFM outperforms baseline models on the CTR dataset.

The model's optimization process is effectively visualized.

Ablation studies highlight the importance of feature mixers.

Abstract

Track one of CTI competition is on click-through rate (CTR) prediction. The dataset contains millions of records and each field-wise feature in a record consists of hashed integers for privacy. For this task, the keys of network-based methods might be type-wise feature extraction and information fusion across different fields. Multi-layer perceptrons (MLPs) are able to extract field feature, but could not efficiently fuse features. Motivated by the natural fusion characteristic of cross attention and the efficiency of transformer-based structures, we propose simple plug-in mixers for field/type-wise feature fusion, and thus construct an field&type-wise ensemble model, namely EMOFM (Ensemble MLP mOdel with Feature-based Mixers). In the experiments, the proposed model is evaluated on the dataset, the optimization process is visualized and ablation studies are explored. It is shown that EMOFM outperforms compared baselines. In the end, we discuss on future work. WARNING: The comparison might not be fair enough since the proposed method is designed for this data in particular while compared methods are not. For example, EMOFM especially takes different types of interactions into consideration while others do not. Anyway, we do hope that the ideas inside our method could help other developers/learners/researchers/thinkers and so on.