Joint Max Margin and Semantic Features for Continuous Event Detection in Complex Scenes

TL;DR

This paper introduces a novel approach combining semantic and temporal features with max-margin classifiers for continuous event detection in videos, achieving state-of-the-art results across multiple challenging datasets.

Contribution

It presents a new model integrating semantic and temporal features with max-margin classifiers for continuous event detection, addressing limitations of local patch-based methods.

Findings

Achieved state-of-the-art performance on UCF 101, MPII Cooking, and Hollywood datasets.

Demonstrated robustness in detecting events with unknown start and end points.

Applicable to both labeled and unlabeled datasets.

Abstract

In this paper the problem of complex event detection in the continuous domain (i.e. events with unknown starting and ending locations) is addressed. Existing event detection methods are limited to features that are extracted from the local spatial or spatio-temporal patches from the videos. However, this makes the model vulnerable to the events with similar concepts e.g. "Open drawer" and "Open cupboard". In this work, in order to address the aforementioned limitations we present a novel model based on the combination of semantic and temporal features extracted from video frames. We train a max-margin classifier on top of the extracted features in an adaptive framework that is able to detect the events with unknown starting and ending locations. Our model is based on the Bidirectional Region Neural Network and large margin Structural Output SVM. The generality of our model allows it to be simply applied to different labeled and unlabeled datasets. We finally test our algorithm on three challenging datasets, "UCF 101-Action Recognition", "MPII Cooking Activities" and "Hollywood", and we report state-of-the-art performance.