A Review of Machine Learning Methods Applied to Video Analysis Systems

TL;DR

This survey reviews machine learning techniques for video analysis, focusing on deep learning methods, dataset challenges, and approaches that reduce labeled data requirements, including self-supervised, semi-supervised, active, and zero-shot learning.

Contribution

It provides a comprehensive overview of recent machine learning methods tailored for video analysis, emphasizing techniques that operate effectively with limited labeled data and real-world complexities.

Findings

Deep learning methods vary in performance across datasets.

Low-parameter models are effective for single-activity detection.

Modern techniques reduce the need for extensive labeled data.

Abstract

The paper provides a survey of the development of machine-learning techniques for video analysis. The survey provides a summary of the most popular deep learning methods used for human activity recognition. We discuss how popular architectures perform on standard datasets and highlight the differences from real-life datasets dominated by multiple activities performed by multiple participants over long periods. For real-life datasets, we describe the use of low-parameter models (with 200X or 1,000X fewer parameters) that are trained to detect a single activity after the relevant objects have been successfully detected. Our survey then turns to a summary of machine learning methods that are specifically developed for working with a small number of labeled video samples. Our goal here is to describe modern techniques that are specifically designed so as to minimize the amount of ground truth that is needed for training and testing video analysis systems. We provide summaries of the development of self-supervised learning, semi-supervised learning, active learning, and zero-shot learning for applications in video analysis. For each method, we provide representative examples.