Integrating Annotations into the Design Process for Sonifications and Physicalizations

TL;DR

This paper explores how annotations, crucial in visualizations, can be integrated into the design of sonifications and physicalizations to improve interpretability and address the lack of formal conventions in these emergent data representations.

Contribution

It introduces methods for incorporating annotations into sonification and physicalization design processes, adapting visualization techniques to non-visual modalities.

Findings

Annotations are integral to data encoding in sonifications and physicalizations.

Designers adapt visualization annotation techniques for sonic and physical data representations.

Annotations enhance interpretability of emergent data representations.

Abstract

Annotations are a critical component of visualizations, helping viewers interpret the visual representation and highlighting critical data insights. Despite their significant role, we lack an understanding of how annotations can be incorporated into other data representations, such as physicalizations and sonifications. Given the emergent nature of these representations, sonifications, and physicalizations lack formalized conventions (e.g., design space, vocabulary) that can introduce challenges for audiences to interpret the intended data encoding. To address this challenge, this work focuses on how annotations can be more tightly integrated into the design process of creating sonifications and physicalizations. In an exploratory study with 13 designers, we explore how visualization annotation techniques can be adapted to sonic and physical modalities. Our work highlights how annotations for sonification and physicalizations are inseparable from their data encodings.