Denoising of photogrammetric dummy head ear point clouds for individual Head-Related Transfer Functions computation

TL;DR

This paper explores the use of deep neural networks to denoise photogrammetric ear scans, improving the accuracy of head-related transfer functions for realistic virtual audio rendering.

Contribution

It introduces a DNN-based denoising approach tailored for photogrammetric ear scans and benchmarks its performance against classical methods, advancing HRTF computation accuracy.

Findings

DNNs marginally improve HRTF accuracy from noisy scans

Correlation analysis identifies key geometric and HRTF metrics

Enhanced DNN retraining further reduces deviation levels

Abstract

Individual Head-Related Transfer Functions (HRTFs), crucial for realistic virtual audio rendering, can be efficiently numerically computed from precise three-dimensional head and ear scans. While photogrammetry scanning is promising, it generally lacks accuracy, leading to HRTFs showing significant perceptual deviation from reference data, mainly due to scanning errors affecting the most occluded pinna structures. This paper examines the application of Deep Neural Networks (DNNs) for denoising photogrammetric ear scans. Several DNNs, fine-tuned on pinna samples corrupted with synthetic error modelled to mimic that observed in photogrammetric dummy head scans, are tested and benchmarked against a classical denoising method. One DNN is further modified and retrained to enhance its denoising performance. The comparison of HRTFs derived from original and denoised scans against reference data shows that the best-performing DNN marginally reduces the deviation of photogrammetric dummy head HRTFs to levels closer to accurately measured ones. Additionally, correlation analysis between geometric and HRTF metrics, computed on the scanned point clouds and their corresponding HRTFs, is used to identify key measures for evaluating the deviation between target and reference scans. These findings are expected to guide the selection of relevant loss functions and foster improvements in this and similar DNN models.