Policy Recognition in the Abstract Hidden Markov Model

TL;DR

This paper introduces the Abstract Hidden Markov Model (AHMM), a new probabilistic framework for online plan recognition in uncertain, multi-level domains, and demonstrates its effectiveness with a video surveillance application.

Contribution

The paper presents the AHMM, a novel stochastic process with a dynamic Bayesian network structure, and develops an efficient hybrid inference method using Rao-Blackwellised Particle Filter for plan recognition.

Findings

AHMM effectively models multi-level plan execution.

Hybrid inference scales well with plan hierarchy levels.

Application demonstrates improved behavior recognition accuracy.

Abstract

In this paper, we present a method for recognising an agent's behaviour in dynamic, noisy, uncertain domains, and across multiple levels of abstraction. We term this problem on-line plan recognition under uncertainty and view it generally as probabilistic inference on the stochastic process representing the execution of the agent's plan. Our contributions in this paper are twofold. In terms of probabilistic inference, we introduce the Abstract Hidden Markov Model (AHMM), a novel type of stochastic processes, provide its dynamic Bayesian network (DBN) structure and analyse the properties of this network. We then describe an application of the Rao-Blackwellised Particle Filter to the AHMM which allows us to construct an efficient, hybrid inference method for this model. In terms of plan recognition, we propose a novel plan recognition framework based on the AHMM as the plan execution model. The Rao-Blackwellised hybrid inference for AHMM can take advantage of the independence properties inherent in a model of plan execution, leading to an algorithm for online probabilistic plan recognition that scales well with the number of levels in the plan hierarchy. This illustrates that while stochastic models for plan execution can be complex, they exhibit special structures which, if exploited, can lead to efficient plan recognition algorithms. We demonstrate the usefulness of the AHMM framework via a behaviour recognition system in a complex spatial environment using distributed video surveillance data.