Activities of Daily Living Indexing by Hierarchical HMM for Dementia Diagnostics

TL;DR

This paper introduces a hierarchical HMM-based method for indexing activities in wearable camera videos to assist in early dementia diagnosis by enabling efficient navigation and analysis of daily activities.

Contribution

It proposes a novel two-step analysis combining motion-based segmentation with multimodal HMM classification for activity indexing in challenging video data.

Findings

Effective segmentation based on apparent motion.

Successful multimodal activity classification.

Good performance demonstrated on real patient data.

Abstract

This paper presents a method for indexing human ac- tivities in videos captured from a wearable camera being worn by patients, for studies of progression of the dementia diseases. Our method aims to produce indexes to facilitate the navigation throughout the individual video recordings, which could help doctors search for early signs of the dis- ease in the activities of daily living. The recorded videos have strong motion and sharp lighting changes, inducing noise for the analysis. The proposed approach is based on a two steps analysis. First, we propose a new approach to segment this type of video, based on apparent motion. Each segment is characterized by two original motion de- scriptors, as well as color, and audio descriptors. Second, a Hidden-Markov Model formulation is used to merge the multimodal audio and video features, and classify the test segments. Experiments show the good properties of the ap- proach on real data.