Utility Change Point Detection in Online Social Media: A Revealed Preference Framework

TL;DR

This paper introduces a framework for detecting change points in utility maximization behavior on social media, using revealed preferences, noise-robust methods, and dimensionality reduction, with applications to real datasets like Yahoo! Tech Buzz and YouTube.

Contribution

It develops a novel change point detection method based on revealed preferences, incorporating noise handling and dimensionality reduction for large-scale social media data.

Findings

Change points in utility functions can be detected from online search behavior.

Recovered utility functions exhibit strategic substitute behavior.

Dimensionality reduction enables utility analysis and traffic prediction in large datasets.

Abstract

This paper deals with change detection of utility maximization behaviour in online social media. Such changes occur due to the effect of marketing, advertising, or changes in ground truth. First, we use the revealed preference framework to detect the unknown time point (change point) at which the utility function changed. We derive necessary and sufficient conditions for detecting the change point. Second, in the presence of noisy measurements, we propose a method to detect the change point and construct a decision test. Also, an optimization criteria is provided to recover the linear perturbation coefficients. Finally, to reduce the computational cost, a dimensionality reduction algorithm using Johnson-Lindenstrauss transform is presented. The results developed are illustrated on two real datasets: Yahoo! Tech Buzz dataset and Youstatanalyzer dataset. By using the results developed in the paper, several useful insights can be gleaned from these data sets. First, the changes in ground truth affecting the utility of the agent can be detected by utility maximization behaviour in online search. Second, the recovered utility functions satisfy the single crossing property indicating strategic substitute behaviour in online search. Third, due to the large number of videos in YouTube, the utility maximization behaviour was verified through the dimensionality reduction algorithm. Finally, using the utility function recovered in the lower dimension, we devise an algorithm to predict total traffic in YouTube.