Alchymical Mirror: Real-time Interactive Sound- and Simple Motion-Tracking Set of Jitter/Max/MSP Patches

TL;DR

This paper presents a real-time interactive system using Jitter/Max/MSP patches that simulates an alchemical process through motion and sound tracking, creating visual transformations and feedback effects.

Contribution

It introduces a novel real-time multimedia setup combining motion and sound tracking with alchemical visual effects using a lattice-based programming model.

Findings

Successful real-time visualization of hand movements as sparkles

Effective integration of sound stability with visual transformations

Interactive system demonstrating alchemical transmutation effects

Abstract

This document supplements an experimental Jitter / Max/MSP collection of implementation patches that set its goal to simulate an alchemical process for a person standing in front of a mirror-like screen while interacting with it. The work involved takes some patience and has three stages to go through. At the final stage the "alchemist" in the mirror wearing sharp-colored gloves (for motion tracking) is to extract the final ultimate shining sparkle (FFT-based visualization) in the nexus of the hands. The more the hands are apart, the large the sparkle should be. Moving hands around should make the sparkle follow. To achieve the desired visual effect and the feedback mechanism, the Jitter lattice-based intensional programming model is used to work on 4-dimensional (A+R+G+B) video matrices and sound signals in order to apply some well-known alchemical techniques to the video at real-time to get a mirror effect and accompanying transmutation and transformation stages of the video based on the stability of the sound produced for some duration of time in real-time. There is an accompanying video of the result with the interaction with the tool and the corresponding programming patches.