TSDCRF: Balancing Privacy and Multi-Object Tracking via Time-Series CRF and Normalized Control Penalty

TL;DR

TSDCRF is a novel framework that enhances multi-object tracking privacy by integrating differential privacy, a normalized control penalty, and a time-series CRF to stabilize associations and reduce ID switches, while maintaining tracking accuracy.

Contribution

It introduces a privacy-preserving multi-object tracking method combining differential privacy, a control penalty, and a dynamic CRF, improving privacy-utility balance and robustness against trajectory hijacking.

Findings

Outperforms prior methods in privacy-utility trade-off

Reduces ID switches and trajectory deviations

Maintains tracking accuracy under privacy constraints

Abstract

Multi-object tracking in video often requires appearance or location cues that can reveal sensitive identity information, while adding privacy-preserving noise typically disrupts cross-frame association and causes ID switches or target loss. We propose TSDCRF, a plug-in refinement framework that balances privacy and tracking by combining three components: (i) $(\varepsilon,\delta)$-differential privacy via calibrated Gaussian noise on sensitive regions under a configurable privacy budget; (ii) a Normalized Control Penalty (NCP) that down-weights unstable or conflicting class predictions before noise injection to stabilize association; and (iii) a time-series dynamic conditional random field (DCRF) that enforces temporal consistency and corrects trajectory deviation after noise, mitigating ID switches and resilience to trajectory hijacking. The pipeline is agnostic to the choice of detector and tracker (e.g., YOLOv4 and DeepSORT). We evaluate on MOT16, MOT17, Cityscapes, and KITTI. Results show that TSDCRF achieves a better privacy--utility trade-off than white noise and prior methods (NTPD, PPDTSA): lower KL-divergence shift, lower tracking RMSE, and improved robustness under trajectory hijacking while preserving privacy. Source code in https://github.com/mabo1215/TSDCRF.git