Modelling Customer Trajectories with Reinforcement Learning for Practical Retail Insights

TL;DR

This paper introduces a reinforcement learning framework for modeling customer trajectories in retail spaces, offering more accurate insights than traditional heuristics and enabling better store layout optimization.

Contribution

The authors develop an agent-based RL model that predicts customer movement more accurately than heuristics, facilitating practical retail insights and layout decisions.

Findings

RL-generated trajectories closely match real customer behaviour

RL-based predictions improve impulse purchase rate estimates

Reinforcement learning enables profitable product repositioning decisions

Abstract

Understanding customer movement within retail spaces is essential for optimizing store layouts. Real-world trajectory data can provide highly accurate insights, but collecting it is costly and often infeasible for many retailers. Heuristics such as Travelling Salesman Problem (TSP) and Probabilistic Nearest Neighbours (PNN) are commonly used as inexpensive approximations, but actual customer trajectories deviate by an average of 28% from shortest paths, highlighting a tradeoff between accuracy and practicality. We propose an agent-based modelling framework that casts customer trajectory prediction as a maximum entropy reinforcement learning (RL) problem, balancing reward maximization with stochasticity to better reflect customers with bounded rationality. Using real-world trajectory data from a convenience store, we show that RL-generated trajectories align more closely with customer behaviour than TSP and PNN, providing more accurate estimates of impulse purchase rates and shelf traffic densities. Furthermore, only RL-based predictions yield repositioning decisions for impulse products that align with those derived from actual trajectory data, resulting in comparable estimated profit gains. Our work demonstrates that RL provides a practical, behaviourally grounded alternative that bridges the gap between oversimplified heuristics and data-intensive approaches, making accurate layout optimization more accessible. To encourage further research, the source code is available on GitHub.