Learning to detect video events from zero or very few video examples

TL;DR

This paper introduces methods for detecting high-level video events using only textual descriptions or very few positive examples, leveraging a novel learning framework and an extended SVM to improve detection accuracy.

Contribution

It proposes a new framework for event detection from text alone and extends SVM to utilize related videos with minimal positive samples, advancing zero-shot and few-shot video event detection.

Findings

Effective detection from textual descriptions demonstrated on TRECVID MED 2014

Extended SVM improves performance with scarce positive samples

Insights into design choices for zero-shot and few-shot learning in videos

Abstract

In this work we deal with the problem of high-level event detection in video. Specifically, we study the challenging problems of i) learning to detect video events from solely a textual description of the event, without using any positive video examples, and ii) additionally exploiting very few positive training samples together with a small number of ``related'' videos. For learning only from an event's textual description, we first identify a general learning framework and then study the impact of different design choices for various stages of this framework. For additionally learning from example videos, when true positive training samples are scarce, we employ an extension of the Support Vector Machine that allows us to exploit ``related'' event videos by automatically introducing different weights for subsets of the videos in the overall training set. Experimental evaluations performed on the large-scale TRECVID MED 2014 video dataset provide insight on the effectiveness of the proposed methods.