Decentralized, not Dehumanized in the Metaverse: Bringing Utility to NFTs through Multimodal Interaction

TL;DR

This paper explores how multimodal interaction can enhance NFTs by integrating human behavior data, promoting decentralization and expanding their artistic potential beyond static content.

Contribution

It proposes a novel framework combining AI multimodal processing with human behavior data to create interactive, generative NFTs that foster decentralization and richer user engagement.

Findings

Multimodal interaction is under-studied in mainstream NFT art.

Integrating human behavior data enables dynamic, generative NFT units.

The framework demonstrates expanded artistic possibilities beyond static content.

Abstract

User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behaviour. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We propose a framework that combines a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behaviour data into generative NFT units, as "multimodal interactive NFT". Our work displays the possibilities of NFTs in the art world, beyond the traditional 2D and 3D static content.