Cross-Modal Terrains: Navigating Sonic Space through Haptic Feedback

TL;DR

This paper introduces a novel method for creating virtual haptic terrains using visual representations and basis functions, enabling immersive sonic and tactile experiences through force-feedback controllers.

Contribution

It presents a new approach to generating multimodal terrains for haptic feedback, diverging from traditional audio-haptic models by using visual textures as a basis for force feedback.

Findings

Effective generation of virtual terrains using basis functions

Real-time haptic feedback varies with terrain grayscale values

Potential for engaging musical performances through this method

Abstract

This paper explores the idea of using virtual textural terrains as a means of generating haptic profiles for force-feedback controllers. This approach breaks from the para-digm established within audio-haptic research over the last few decades where physical models within virtual environments are designed to transduce gesture into sonic output. We outline a method for generating multimodal terrains using basis functions, which are rendered into monochromatic visual representations for inspection. This visual terrain is traversed using a haptic controller, the NovInt Falcon, which in turn receives force information based on the grayscale value of its location in this virtual space. As the image is traversed by a performer the levels of resistance vary, and the image is realized as a physical terrain. We discuss the potential of this approach to afford engaging musical experiences for both the performer and the audience as iterated through numerous performances.