Analysis of E-commerce Ranking Signals via Signal Temporal Logic

TL;DR

This paper introduces a novel approach using Signal Temporal Logic to analyze and characterize document ranking behaviors over time, revealing patterns that impact learning to rank models.

Contribution

The work applies STL formalism to ranking signals, enabling formalization and detection of document behaviors in learning to rank models.

Findings

Detection of patterns like cold start, warm start, and spikes.

Insights into how behaviors affect ranking model performance.

Validation on a dataset of 100K product signals.

Abstract

The timed position of documents retrieved by learning to rank models can be seen as signals. Signals carry useful information such as drop or rise of documents over time or user behaviors. In this work, we propose to use the logic formalism called Signal Temporal Logic (STL) to characterize document behaviors in ranking accordingly to the specified formulas. Our analysis shows that interesting document behaviors can be easily formalized and detected thanks to STL formulas. We validate our idea on a dataset of 100K product signals. Through the presented framework, we uncover interesting patterns, such as cold start, warm start, spikes, and inspect how they affect our learning to ranks models.