Learning to Infer User Hidden States for Online Sequential Advertising

TL;DR

This paper introduces DISA, a method that infers consumers' hidden purchase intents as latent states within a POMDP framework, enhancing interpretability and performance in online sequential advertising through large-scale experiments.

Contribution

The paper proposes a novel approach to infer unobservable consumer intents as latent states using POMDP, improving interpretability and effectiveness in online advertising strategies.

Findings

DISA outperforms baseline methods in offline and online experiments.

Inferred hidden states align with consumer behavior patterns.

The approach enhances understanding and optimization of advertising strategies.

Abstract

To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important. The lack of interpretability in existing deep reinforcement learning methods makes it not easy to understand, diagnose and further optimize the strategy. In this paper, we propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer's purchase intent which is, however, unobservable (called hidden states). In this paper, we model this intention as a latent variable and formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the underlying intents are inferred based on the observable behaviors. Large-scale industrial offline and online experiments demonstrate our method's superior performance over several baselines. The inferred hidden states are analyzed, and the results prove the rationality of our inference.