A Logic Programming Approach to Activity Recognition

TL;DR

This paper presents a logic programming system that recognizes complex human activities from video by using Event Calculus to model temporal constraints, demonstrated on surveillance video datasets.

Contribution

It introduces a novel logic programming approach employing Event Calculus for activity recognition from symbolic video data, enhancing expressiveness and interpretability.

Findings

Effective recognition of long-term activities from short-term detections

Demonstrated system performance on surveillance video dataset

Expressive representation of complex activities using Event Calculus

Abstract

We have been developing a system for recognising human activity given a symbolic representation of video content. The input of our system is a set of time-stamped short-term activities detected on video frames. The output of our system is a set of recognised long-term activities, which are pre-defined temporal combinations of short-term activities. The constraints on the short-term activities that, if satisfied, lead to the recognition of a long-term activity, are expressed using a dialect of the Event Calculus. We illustrate the expressiveness of the dialect by showing the representation of several typical complex activities. Furthermore, we present a detailed evaluation of the system through experimentation on a benchmark dataset of surveillance videos.