A Large-scale Distributed Video Parsing and Evaluation Platform

TL;DR

This paper presents a scalable, extensible platform for large-scale video parsing and evaluation in surveillance, integrating big data technologies and user feedback to improve recognition tasks.

Contribution

It introduces a novel distributed platform combining Spark Streaming, Kafka, and HDFS for scalable video analysis with a web interface for performance evaluation.

Findings

Supports large-scale surveillance video processing

Enables incremental algorithm development

Provides user feedback for system performance

Abstract

Visual surveillance systems have become one of the largest data sources of Big Visual Data in real world. However, existing systems for video analysis still lack the ability to handle the problems of scalability, expansibility and error-prone, though great advances have been achieved in a number of visual recognition tasks and surveillance applications, e.g., pedestrian/vehicle detection, people/vehicle counting. Moreover, few algorithms explore the specific values/characteristics in large-scale surveillance videos. To address these problems in large-scale video analysis, we develop a scalable video parsing and evaluation platform through combining some advanced techniques for Big Data processing, including Spark Streaming, Kafka and Hadoop Distributed Filesystem (HDFS). Also, a Web User Interface is designed in the system, to collect users' degrees of satisfaction on the recognition tasks so as to evaluate the performance of the whole system. Furthermore, the highly extensible platform running on the long-term surveillance videos makes it possible to develop more intelligent incremental algorithms to enhance the performance of various visual recognition tasks.