REVAMP$^2$T: Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking

TL;DR

REVAMP$^2$T is a novel real-time, privacy-aware multi-camera pedestrian tracking system that integrates deep learning algorithms with edge IoT infrastructure, achieving significant improvements in accuracy and efficiency.

Contribution

It introduces a unified computer vision pipeline and IoT system architecture for privacy-preserving, real-time multi-camera pedestrian tracking at the edge.

Findings

Outperforms state-of-the-art by up to 13 times in the new  metric.

Provides a privacy-aware system avoiding facial recognition.

Achieves high hardware utilization and system-wide re-identification.

Abstract

This article presents REVAMP$^2$T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacy-built-in decentralized situational awareness. REVAMP$^2$T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP$^2$T proposes a unified integrated computer vision pipeline for detection, re-identification, and tracking across multiple cameras without the need for storing the streaming data. At the same time, it avoids facial recognition, and tracks and re-identifies pedestrians based on their key features at runtime. On the IoT system side, REVAMP$^2$T provides infrastructure to maximize hardware utilization on the edge, orchestrates global communications, and provides system-wide re-identification, without the use of personally identifiable information, for a distributed IoT network. For the results and evaluation, this article also proposes a new metric, Accuracy$\cdot$Efficiency (\AE), for holistic evaluation of IoT systems for real-time video analytics based on accuracy, performance, and power efficiency. REVAMP$^2$T outperforms current state-of-the-art by as much as thirteen-fold \AE~improvement.