Team voyTECH: User Activity Modeling with Boosting Trees

TL;DR

This paper presents a successful approach for predicting Twitch user subscriptions using boosting trees, highlighting the effectiveness of target-encoding for high-cardinality features and demonstrating improved modeling of user activity over content-based methods.

Contribution

The paper introduces a novel connection between target-encodings and boosting trees for high-cardinality categorical features in user activity modeling.

Findings

Modeling user activity with boosting trees predicts subscriptions effectively.

Target-encoding enhances the performance of boosting trees on high-cardinality data.

User activity modeling outperforms content-based approaches when encoded properly.

Abstract

This paper describes our winning solution for the ECML-PKDD ChAT Discovery Challenge 2020. We show that whether or not a Twitch user has subscribed to a channel can be well predicted by modeling user activity with boosting trees. We introduce the connection between target-encodings and boosting trees in the context of high cardinality categoricals and find that modeling user activity is more powerful then direct modeling of content when encoded properly and combined with a suitable optimization approach.