Material fingerprinting: predicting human perception of material appearance through psychophysical analysis and neural networks

TL;DR

This paper introduces a new method to predict how humans perceive material appearances using images and machine learning, enabling better digital representation of materials.

Contribution

The novel 'visual fingerprint' links image data to human-perceived material attributes using psychophysical studies and neural networks.

Findings

Human ratings of material appearance attributes were mapped using over 110,000 responses across 347 materials.

A predictive model using CLIP features and a multi-layer perceptron accurately forecasts human perception from just two images.

The framework enables intuitive comparisons and retrievals of material appearances in digital applications.

Abstract

Digital representation of materials is crucial in fields such as virtual reality, industrial design and quality control. However, predicting human perception of materials from image data is challenging due to the complexity of material appearances and the intricacies of human vision. This study introduces a perceptual representation termed the ‘visual fingerprint’, linking image-based measurements of materials to intuitive, human-understandable attributes. We conducted psychophysical studies using standardized video sequences of 347 diverse real-world materials, including fabrics and wood, selected to encompass a broad spectrum of textures, colours and reflective properties. Sixteen key appearance attributes were identified, and over 110 000 human ratings were collected to map perceptual attributes across material categories. By integrating CLIP-derived image features with a multi-layer perceptron model, we developed a predictive framework for material perception. Our results demonstrate that human judgements of appearance and similarity can be accurately predicted using only two images of a material. This work offers a practical and interpretable approach to material representation, enabling intuitive comparisons and retrievals in applications where material appearance is crucial. The proposed material fingerprint and its prediction directly from image data represent a significant step towards simplifying the understanding and interoperability of material properties in diverse digital environments.

Figures