Benchmarking Jetson Edge Devices with an End-to-end Video-based Anomaly Detection System

TL;DR

This paper benchmarks Jetson edge devices by deploying an end-to-end video anomaly detection system, demonstrating performance improvements and energy efficiency gains across different Jetson models using deep learning models and NVIDIA software tools.

Contribution

It presents a comprehensive deployment and benchmarking of a video-based anomaly detection system on multiple Jetson edge devices, highlighting performance and energy efficiency improvements.

Findings

Achieves 47.56 FPS inference speed on Jetson devices.

System uses only 3.11 GB RAM during operation.

15% better performance with 50% less energy consumption compared to previous Jetson models.

Abstract

Innovative enhancement in embedded system platforms, specifically hardware accelerations, significantly influence the application of deep learning in real-world scenarios. These innovations translate human labor efforts into automated intelligent systems employed in various areas such as autonomous driving, robotics, Internet-of-Things (IoT), and numerous other impactful applications. NVIDIA's Jetson platform is one of the pioneers in offering optimal performance regarding energy efficiency and throughput in the execution of deep learning algorithms. Previously, most benchmarking analysis was based on 2D images with a single deep learning model for each comparison result. In this paper, we implement an end-to-end video-based crime-scene anomaly detection system inputting from surveillance videos and the system is deployed and completely operates on multiple Jetson edge devices (Nano, AGX Xavier, Orin Nano). The comparison analysis includes the integration of Torch-TensorRT as a software developer kit from NVIDIA for the model performance optimisation. The system is built based on the PySlowfast open-source project from Facebook as the coding template. The end-to-end system process comprises the videos from camera, data preprocessing pipeline, feature extractor and the anomaly detection. We provide the experience of an AI-based system deployment on various Jetson Edge devices with Docker technology. Regarding anomaly detectors, a weakly supervised video-based deep learning model called Robust Temporal Feature Magnitude Learning (RTFM) is applied in the system. The approach system reaches 47.56 frames per second (FPS) inference speed on a Jetson edge device with only 3.11 GB RAM usage total. We also discover the promising Jetson device that the AI system achieves 15% better performance than the previous version of Jetson devices while consuming 50% less energy power.