Motion shapes for sound shaping

TL;DR

This paper explores how shapes can represent both body movements that produce sound and the resulting sound features, making them easier to understand and study.

Contribution

It introduces the concept of 'motion shapes' as a unifying framework for understanding sound and movement.

Findings

Shapes can represent dynamic and spectral features of sound.

Motion shapes include both movement paths and body postures involved in sound production.

Shapes help make transient sound and motion features more permanent and analyzable.

Abstract

The focus of this perspective paper is on relationships between sound-producing body motion and corresponding perceived sound features, guided by the idea of shapes as the common denominator of these two domains. The term shape is used to denote graphical-pictorial renderings of phenomena that we perceive or imagine, and may have physical manifestations as tracings on paper or on screen, or as gesticulations, or just as imagined tracings in our minds. Shapes give us intermittent snapshots of unfolding motion and sound fragments, and the point of shapes is to make ephemeral sound and motion features tractable as more permanent objects. Shapes of perceived sound include dynamic, spectral, textural, pitch-related, harmonic, etc. features as shapes, whereas shapes of sound-producing motion include both motion trajectories and postures of sound-producing effectors, i.e., of fingers, hands, arms, etc., or mouth, lips, and tongue.

Figures