Text2Traj2Text: Learning-by-Synthesis Framework for Contextual Captioning of Human Movement Trajectories

TL;DR

This paper introduces Text2Traj2Text, a framework that uses large language models to generate and utilize synthetic data for improving contextual captioning of human movement trajectories in retail environments.

Contribution

It proposes a novel learning-by-synthesis approach leveraging language models to generate diverse, realistic trajectory captions for better customer behavior understanding.

Findings

Outperforms competitive methods in ROUGE and BERT Score metrics.

Models trained on synthetic data generalize well to real human trajectories.

Framework enhances retail applications like targeted advertising and inventory management.

Abstract

This paper presents Text2Traj2Text, a novel learning-by-synthesis framework for captioning possible contexts behind shopper's trajectory data in retail stores. Our work will impact various retail applications that need better customer understanding, such as targeted advertising and inventory management. The key idea is leveraging large language models to synthesize a diverse and realistic collection of contextual captions as well as the corresponding movement trajectories on a store map. Despite learned from fully synthesized data, the captioning model can generalize well to trajectories/captions created by real human subjects. Our systematic evaluation confirmed the effectiveness of the proposed framework over competitive approaches in terms of ROUGE and BERT Score metrics.