Feature Modulation to Improve Struggle Detection in Web Search: A Psychological Approach

TL;DR

This paper introduces a psychological theory-inspired feature modulation method to enhance struggle detection in web search, addressing limitations of effort-based features by considering dynamic psychological states.

Contribution

It proposes a novel feature modulation approach based on reversal theory, improving struggle detection accuracy over existing effort-based methods.

Findings

Significant improvement over state-of-the-art struggle detection methods

Supports the psychological theory with empirical evidence

Enhances understanding of user frustration dynamics

Abstract

Searcher struggle is important feedback to Web search engines. Existing Web search struggle detection methods rely on effort-based features to identify the struggling moments. Their underlying assumption is that the more effort a user spends, the more struggling the user may be. However, recent studies have suggested this simple association might be incorrect. This paper proposes a new feature modulation method for struggle detection and refers to the reversal theory in psychology. The reversal theory (RT) points out that instead of having a static personality trait, people constantly switch between opposite psychological states, complicating the relationship between the efforts they spend and the level of frustration they feel. Supported by the theory, our method modulates the effort-based features based on RT's bi-modal arousal model. Evaluations on week-long Web search logs confirm that the proposed method can statistically significantly improve state-of-the-art struggle detection methods.