Rethinking Voice-Face Correlation: A Geometry View

TL;DR

This paper explores reconstructing 3D facial shapes from voice using anthropometric measurements, revealing specific geometric correlations and offering a new perspective beyond semantic cues.

Contribution

It introduces a novel voice-AM-face paradigm that links voice to face geometry through predictable anthropometric measurements, advancing understanding of voice-face correlations.

Findings

Significant correlations between voice and facial geometry parts.

The proposed method effectively reconstructs 3D faces from voice data.

Provides a new geometric perspective on voice-face relationships.

Abstract

Previous works on voice-face matching and voice-guided face synthesis demonstrate strong correlations between voice and face, but mainly rely on coarse semantic cues such as gender, age, and emotion. In this paper, we aim to investigate the capability of reconstructing the 3D facial shape from voice from a geometry perspective without any semantic information. We propose a voice-anthropometric measurement (AM)-face paradigm, which identifies predictable facial AMs from the voice and uses them to guide 3D face reconstruction. By leveraging AMs as a proxy to link the voice and face geometry, we can eliminate the influence of unpredictable AMs and make the face geometry tractable. Our approach is evaluated on our proposed dataset with ground-truth 3D face scans and corresponding voice recordings, and we find significant correlations between voice and specific parts of the face geometry, such as the nasal cavity and cranium. Our work offers a new perspective on voice-face correlation and can serve as a good empirical study for anthropometry science.