Cross-Modal Terrains: Navigating Sonic Space through Haptic Feedback
Gabriella Isaac, Lauren Hayes, Todd Ingalls

TL;DR
This paper introduces a novel method for creating virtual haptic terrains using visual representations and basis functions, enabling immersive sonic and tactile experiences through force-feedback controllers.
Contribution
It presents a new approach to generating multimodal terrains for haptic feedback, diverging from traditional audio-haptic models by using visual textures as a basis for force feedback.
Findings
Effective generation of virtual terrains using basis functions
Real-time haptic feedback varies with terrain grayscale values
Potential for engaging musical performances through this method
Abstract
This paper explores the idea of using virtual textural terrains as a means of generating haptic profiles for force-feedback controllers. This approach breaks from the para-digm established within audio-haptic research over the last few decades where physical models within virtual environments are designed to transduce gesture into sonic output. We outline a method for generating multimodal terrains using basis functions, which are rendered into monochromatic visual representations for inspection. This visual terrain is traversed using a haptic controller, the NovInt Falcon, which in turn receives force information based on the grayscale value of its location in this virtual space. As the image is traversed by a performer the levels of resistance vary, and the image is realized as a physical terrain. We discuss the potential of this approach to afford engaging musical experiences for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic Technology and Sound Studies · Tactile and Sensory Interactions · Multisensory perception and integration
