Object Detection Approaches to Identifying Hand Images with High Forensic Values

TL;DR

This paper compares machine learning models for hand detection in forensic images, demonstrating that YOLOv8 models outperform other approaches and can significantly aid forensic investigations.

Contribution

The study fine-tunes and evaluates YOLOv8 and transformer-based models for hand detection, showing YOLOv8's superior performance in forensic contexts.

Findings

YOLOv8 models outperform DETR and DETA in accuracy.

YOLOv8 models outperform previous YOLOv3 and YOLOv4 methods.

Application of YOLOv8 reduces forensic analysis time.

Abstract

Forensic science plays a crucial role in legal investigations, and the use of advanced technologies, such as object detection based on machine learning methods, can enhance the efficiency and accuracy of forensic analysis. Human hands are unique and can leave distinct patterns, marks, or prints that can be utilized for forensic examinations. This paper compares various machine learning approaches to hand detection and presents the application results of employing the best-performing model to identify images of significant importance in forensic contexts. We fine-tune YOLOv8 and vision transformer-based object detection models on four hand image datasets, including the 11k hands dataset with our own bounding boxes annotated by a semi-automatic approach. Two YOLOv8 variants, i.e., YOLOv8 nano (YOLOv8n) and YOLOv8 extra-large (YOLOv8x), and two vision transformer variants, i.e., DEtection TRansformer (DETR) and Detection Transformers with Assignment (DETA), are employed for the experiments. Experimental results demonstrate that the YOLOv8 models outperform DETR and DETA on all datasets. The experiments also show that YOLOv8 approaches result in superior performance compared with existing hand detection methods, which were based on YOLOv3 and YOLOv4 models. Applications of our fine-tuned YOLOv8 models for identifying hand images (or frames in a video) with high forensic values produce excellent results, significantly reducing the time required by forensic experts. This implies that our approaches can be implemented effectively for real-world applications in forensics or related fields.