Integrating AIs With Body Tracking Technology for Human Behaviour Analysis: Challenges and Opportunities

TL;DR

This paper explores integrating AI with body tracking technology using commodity depth cameras for human behavior analysis, highlighting challenges and opportunities in creating interactive systems and research tools.

Contribution

It discusses the integration of AI components with body tracking systems, based on practical experience with a remote collaboration system using existing AI models.

Findings

AI-enhanced body tracking improves behavior analysis capabilities

Challenges include orchestrating multiple AI components and engineering complex pipelines

Opportunities for advanced human-computer interaction systems

Abstract

The automated analysis of human behaviour provides many opportunities for the creation of interactive systems and the post-experiment investigations for user studies. Commodity depth cameras offer reasonable body tracking accuracy at a low price point, without the need for users to wear or hold any extra equipment. The resulting systems typically perform body tracking through a dedicated machine learning model, but they can be enhanced with additional AI components providing extra capabilities. This leads to opportunities but also challenges, for example regarding the orchestration of such AI components and the engineering of the resulting tracking pipeline. In this paper, we discuss these elements, based on our experience with the creation of a remote collaboration system across distant wall-sized displays, that we built using existing and readily available building blocks, including AI-based recognition models.