Real-time Embedded Person Detection and Tracking for Shopping Behaviour Analysis

TL;DR

This paper presents a real-time, embedded person detection and tracking system using optimized YOLOv3 on Jetson TX2, enabling store behavior analysis with high accuracy and efficiency in challenging conditions.

Contribution

It introduces a lightweight, real-time detection and tracking solution combining TensorRT-optimized YOLOv3 and optical flow on embedded hardware for shopping environments.

Findings

Achieved 81.59% average precision at 10 FPS

Successfully handled occlusions with sparse optical flow tracking

Generated heat maps for store layout analysis

Abstract

Shopping behaviour analysis through counting and tracking of people in shop-like environments offers valuable information for store operators and provides key insights in the stores layout (e.g. frequently visited spots). Instead of using extra staff for this, automated on-premise solutions are preferred. These automated systems should be cost-effective, preferably on lightweight embedded hardware, work in very challenging situations (e.g. handling occlusions) and preferably work real-time. We solve this challenge by implementing a real-time TensorRT optimized YOLOv3-based pedestrian detector, on a Jetson TX2 hardware platform. By combining the detector with a sparse optical flow tracker we assign a unique ID to each customer and tackle the problem of loosing partially occluded customers. Our detector-tracker based solution achieves an average precision of 81.59% at a processing speed of 10 FPS. Besides valuable statistics, heat maps of frequently visited spots are extracted and used as an overlay on the video stream.