Ghostfinger: a novel platform for fully computational fingertip controllers

TL;DR

Ghostfinger introduces a fully computational fingertip haptic control platform that visualizes dynamic haptic primitives and provides an immersive, tangible interaction experience akin to hologram interaction, with applications in digital musical instruments.

Contribution

The paper presents Ghostfinger, a novel platform combining real-time visualization of haptic primitives with advanced fingertip control for digital musical instruments.

Findings

Real-time visualization of fingertip transducer and haptic primitives.

Enhanced user interaction through visual and symbolic feedback.

Potential applications in digital musical instrument control.

Abstract

We present Ghostfinger, a technology for highly dynamic up/down fingertip haptics and control. The overall user experience offered by the technology can be described as that of tangibly and audibly interacting with a small hologram. More specifically, Ghostfinger implements automatic visualization of the dynamic instantiation/parametrization of algorithmic primitives that together determine the current haptic conditions for fingertip action. Some aspects of this visualization are visuospatial: A floating see-through cursor provides real-time, to-scale display of the fingerpad transducer, as it is being moved by the user. Simultaneously, each haptic primitive instance is represented by a floating block shape, type-colored, variably transparent, and possibly overlapping with other such block shapes. Further aspects of visualization are symbolic: Each instance is also represented by a type symbol, lighting up within a grid if the instance is providing output to the user. We discuss the system's user interface, programming interface, and potential applications. This is done from a general perspective that articulates and emphasizes the uniquely enabling role of the principle of computation in the implementation of new forms of instrumental control of musical sound. Beyond the currently presented technology, this also reflects more broadly on the role of Digital Musical Instruments (DMIs) in NIME.