Indoor Group Activity Recognition using Multi-Layered HMMs

TL;DR

This paper introduces a multi-layered Hidden Markov Model framework for recognizing and classifying group activities from imagery data, leveraging ontologies and different HMM configurations to improve accuracy in surveillance applications.

Contribution

It proposes a novel multi-layered HMM architecture with three implementation variants for effective group activity recognition using ontologies.

Findings

Compared the performance of concatenated, cascaded, and hybrid HMMs

Demonstrated improved activity recognition accuracy

Showed effective discrimination of group activities in surveillance data

Abstract

Discovery and recognition of Group Activities (GA) based on imagery data processing have significant applications in persistent surveillance systems, which play an important role in some Internet services. The process is involved with analysis of sequential imagery data with spatiotemporal associations. Discretion of video imagery requires a proper inference system capable of discriminating and differentiating cohesive observations and interlinking them to known ontologies. We propose an Ontology based GAR with a proper inference model that is capable of identifying and classifying a sequence of events in group activities. A multi-layered Hidden Markov Model (HMM) is proposed to recognize different levels of abstract GA. The multi-layered HMM consists of N layers of HMMs where each layer comprises of M number of HMMs running in parallel. The number of layers depends on the order of information to be extracted. At each layer, by matching and correlating attributes of detected group events, the model attempts to associate sensory observations to known ontology perceptions. This paper demonstrates and compares performance of three different implementation of HMM, namely, concatenated N-HMM, cascaded C-HMM and hybrid H-HMM for building effective multi-layered HMM.