Efficient Feature Interactions with Transformers: Improving User Spending Propensity Predictions in Gaming

TL;DR

This paper introduces a novel transformer-based architecture tailored for modeling complex feature interactions to predict user spending propensity in gaming, outperforming existing models on structured data.

Contribution

We propose a new transformer architecture designed to better capture feature interactions, leading to improved prediction accuracy over existing models.

Findings

Our model surpasses FT-Transformer with 2.5% lower MAE.

Achieves 21.8% reduction in MSE compared to baseline.

Demonstrates effectiveness on structured transaction data.

Abstract

Dream11 is a fantasy sports platform that allows users to create their own virtual teams for real-life sports events. We host multiple sports and matches for our 200M+ user base. In this RMG (real money gaming) setting, users pay an entry amount to participate in various contest products that we provide to users. In our current work, we discuss the problem of predicting the user's propensity to spend in a gaming round, so it can be utilized for various downstream applications. e.g. Upselling users by incentivizing them marginally as per their spending propensity, or personalizing the product listing based on the user's propensity to spend. We aim to model the spending propensity of each user based on past transaction data. In this paper, we benchmark tree-based and deep-learning models that show good results on structured data, and we propose a new architecture change that is specifically designed to capture the rich interactions among the input features. We show that our proposed architecture outperforms the existing models on the task of predicting the user's propensity to spend in a gaming round. Our new transformer model surpasses the state-of-the-art FT-Transformer, improving MAE by 2.5\% and MSE by 21.8\%.