The Computerized Classification of Micro-Motions in the Hand using Waveforms from Mobile Phone

TL;DR

This paper introduces a novel method using mobile phone videos and advanced image processing combined with machine learning to classify micro-motions in the hand, achieving around 92% accuracy.

Contribution

It presents a new approach integrating Eulerian Video Magnification, Skeletonization, and kNN for classifying hand micro-motions from video waveforms.

Findings

Achieved approximately 92% classification accuracy.

Successfully distinguished hand, vein, background, and respiration motions.

Demonstrated the effectiveness of waveforms from video analysis for micro-motion classification.

Abstract

Our hands reveal important information such as the pulsing of our veins which help us determine the blood pressure, tremors indicative of motor control, or neurodegenerative disorders such as Essential Tremor or Parkinson's disease. The Computerized Classification of Micro-Motions in the hand using waveforms from mobile phone videos is a novel method that uses Eulerian Video Magnification, Skeletonization, Heatmapping, and the kNN machine learning model to detect the micro-motions in the human hand, synthesize their waveforms, and classify these. The pre-processing is achieved by using Eulerian Video Magnification, Skeletonization, and Heat-mapping to magnify the micro-motions, landmark essential features of the hand, and determine the extent of motion, respectively. Following pre-processing, the visible motions are manually labeled by appropriately grouping pixels to represent a particular label correctly. These labeled motions of the pixels are converted into waveforms. Finally, these waveforms are classified into four categories - hand or finger movements, vein movement, background motion, and movement of the rest of the body due to respiration using the kNN model. The final accuracy obtained was around 92 percent.