3D Visual Tracking to Quantify Physical Contact Interactions in Human-to-Human Touch

TL;DR

This paper introduces a 3D visual tracking system using a depth camera to precisely measure skin-to-skin contact attributes during human interactions, enabling detailed analysis of social touch behaviors.

Contribution

It develops a novel interference-free 3D tracking method that captures high-resolution contact attributes between hands and forearms, advancing measurement capabilities in social touch analysis.

Findings

System can distinguish different social messages and gestures based on contact attributes.

Measurement accuracy validated against independent sensors like electromagnetic trackers and pressure mats.

Applicable to various contexts such as maternal care and massage therapy.

Abstract

Across a plethora of social situations, we touch others in natural and intuitive ways to share thoughts and emotions, such as tapping to get one's attention or caressing to soothe one's anxiety. A deeper understanding of these human-to-human interactions will require, in part, the precise measurement of skin-to-skin physical contact. Among prior efforts, each measurement approach exhibits certain constraints, e.g., motion trackers do not capture the precise shape of skin surfaces, while pressure sensors impede direct skin contact. In contrast, this work develops an interference-free 3D visual tracking system using a depth camera to measure the contact attributes between the bare hand of a toucher and the forearm of a receiver. The toucher's hand is tracked as a posed and positioned mesh by fitting a hand model to detected 3D joints, whereas the forearm is extracted as a detailed 3D surface. Based on a contact model of point clouds, the spatiotemporal contact changes are decomposed as six high-resolution time-series attributes, i.e., contact area, indentation depth, absolute velocity, and three orthogonal velocity components, together with contact duration. To examine the system's capabilities and limitations, two experiments were performed. First, to evaluate its ability to discern human touches, one person delivered cued social messages, e.g., happiness, anger, sympathy, to another using their preferred gestures. The results indicated that messages, gestures, and the individual touchers, were readily discerned from their contact attributes. Second, the measurement accuracy was validated against independent devices, including an electromagnetic tracker, pressure mat, and laser sensor. While validated here in the context of social communication, this system is extendable to human touch interactions such as maternal care of infants and massage therapy.