Paint by Odor: An Exploration of Odor Visualization through Large Language Model and Generative AI

TL;DR

This paper presents 'Paint by Odor', a novel pipeline using large language models and generative AI to automatically convert olfactory perceptions into visual representations, advancing odor visualization beyond manual methods.

Contribution

It introduces a new automated approach for odor visualization leveraging LLMs and generative AI, bridging olfactory perception with visual generation, and explores the effects of language descriptions and styles.

Findings

LLMs can approximate human olfactory descriptions.

Generated odor images are influenced by language and style.

The approach enables automatic odor visualization without human input.

Abstract

Odor visualization translates odor information and perception into visual outcomes and arouses the corresponding olfactory synesthesia, surpassing the spatial limitation that odors can only be perceived where they are present. Traditional odor visualization has typically relied on unidimensional mappings, such as odor-to-color associations, and has required extensive manual design efforts. However, the advent of generative AI (Gen AI) and large language models (LLMs) presents a new opportunity for automatic odor visualization. Nonetheless, gaps remain in bridging olfactory perception with generative tools to produce odor images. To address these gaps, this paper introduces Paint by Odor, a pipeline that leverages Gen AI and LLMs to transform olfactory perceptions into rich, aesthetically engaging visual representations. Two experiments were conducted, where 30 participants smelled real-world odors and provided descriptive data and 28 participants evaluated 560 generated odor images through seven systematically designed prompts. Our findings explored the capability of LLMs in producing olfactory perception by comparing it with human responses and revealed the underlying mechanisms and effects of language-based descriptions and several abstraction styles on odor visualization. Our work further discussed the possibility of automatic odor visualization without human participation. These explorations and results have bridged the research gap in odor visualization using LLMs and Gen AI, offering valuable design insights and various possibilities for future applications.