Deferred is Better: A Framework for Multi-Granularity Deferred Interaction of Heterogeneous Features

TL;DR

This paper introduces MGDIN, a novel framework that adaptively defers feature interactions in CTR models by grouping features based on information density and gradually unmasking them, improving robustness and representation learning.

Contribution

The paper proposes a multi-granularity deferred interaction framework that adaptively manages feature heterogeneity in CTR models, enhancing learning efficiency and robustness.

Findings

Improved CTR prediction accuracy over baseline models.

Effective handling of feature heterogeneity through deferred interaction.

Enhanced model robustness and interpretability.

Abstract

Click-through rate (CTR) prediction models estimates the probability of a user-item click by modeling interactions across a vast feature space. A fundamental yet often overlooked challenge is the inherent heterogeneity of these features: their sparsity and information content vary dramatically. For instance, categorical features like item IDs are extremely sparse, whereas numerical features like item price are relatively dense. Prevailing CTR models have largely ignored this heterogeneity, employing a uniform feature interaction strategy that inputs all features into the interaction layers simultaneously. This approach is suboptimal, as the premature introduction of low-information features can inject significant noise and mask the signals from information-rich features, which leads to model collapse and hinders the learning of robust representations. To address the above challenge, we propose a Multi-Granularity Information-Aware Deferred Interaction Network (MGDIN), which adaptively defers the introduction of features into the feature interaction process. MGDIN's core mechanism operates in two stages: First, it employs a multi-granularity feature grouping strategy to partition the raw features into distinct groups with more homogeneous information density in different granularities, thereby mitigating the effects of extreme individual feature sparsity and enabling the model to capture feature interactions from diverse perspectives. Second, a delayed interaction mechanism is implemented through a hierarchical masking strategy, which governs when and how each group participates by masking low-information groups in the early layers and progressively unmasking them as the network deepens. This deferred introduction allows the model to establish a robust understanding based on high-information features before gradually incorporating sparser information from other groups...