Unsupervised Skin Feature Tracking with Deep Neural Networks

TL;DR

This paper introduces an unsupervised deep learning pipeline for skin feature tracking that achieves high accuracy without extensive labeled data, outperforming traditional and modern supervised methods in challenging motion scenarios.

Contribution

The paper presents a novel unsupervised autoencoder-based method for skin feature tracking, reducing data needs and improving accuracy over existing supervised and traditional techniques.

Findings

Mean error between 0.6 and 3.3 pixels

Outperforms SIFT, SURF, Lucas Kanade, PIPs++, and CoTracker

Effective in high-motion conditions

Abstract

Facial feature tracking is essential in imaging ballistocardiography for accurate heart rate estimation and enables motor degradation quantification in Parkinson's disease through skin feature tracking. While deep convolutional neural networks have shown remarkable accuracy in tracking tasks, they typically require extensive labeled data for supervised training. Our proposed pipeline employs a convolutional stacked autoencoder to match image crops with a reference crop containing the target feature, learning deep feature encodings specific to the object category in an unsupervised manner, thus reducing data requirements. To overcome edge effects making the performance dependent on crop size, we introduced a Gaussian weight on the residual errors of the pixels when calculating the loss function. Training the autoencoder on facial images and validating its performance on manually labeled face and hand videos, our Deep Feature Encodings (DFE) method demonstrated superior tracking accuracy with a mean error ranging from 0.6 to 3.3 pixels, outperforming traditional methods like SIFT, SURF, Lucas Kanade, and the latest transformers like PIPs++ and CoTracker. Overall, our unsupervised learning approach excels in tracking various skin features under significant motion conditions, providing superior feature descriptors for tracking, matching, and image registration compared to both traditional and state-of-the-art supervised learning methods.