A short review and primer on online processing of multiple signal sources in human computer interaction applications

TL;DR

This paper reviews the use of stream processing systems for real-time analysis of multimodal physiological signals in human-computer interaction, emphasizing practical applications and system design considerations.

Contribution

It provides a comprehensive primer on stream processing solutions and analysis techniques tailored for HCI applications involving physiological signals.

Findings

Overview of hardware and software SPS solutions for HCI

Design considerations for analyzing physiological signals

Focus on everyday HCI applications rather than clinical or sports uses

Abstract

The application of psychophysiological in human-computer interaction is a growing field with significant potential for future smart personalised systems. Working in this emerging field requires comprehension of an array of physiological signals and analysis techniques. Stream processing sytems (SPS) are emerging computational platforms that can be utilized in human-computer interaction for real-time analysis of high-volume multimodal signals. Usage of complementary information contained in multiple signals is desireable as it can make HCI systems more robust. In this preprint we review existing software and hardware solutions for HCI-centric stream processing systems. The preprint also includes a brief introduction into SPS design considerations and structure from the perspective of analysing physiological signals. This paper aims to serve as a primer for the novice, enabling rapid familiarisation with the latest core concepts. We put special emphasis on everyday human-computer interface applications to distinguish from the more common clinical or sports uses of psychophysiology. This paper is an extract from a comprehensive review of the entire field of ambulatory psychophysiology, including 12 similar chapters, plus application guidelines and systematic review. Thus any citation should be made using the following reference: B. Cowley, M. Filetti, K. Lukander, J. Torniainen, A. Henelius, L. Ahonen, O. Barral, I. Kosunen, T. Valtonen, M. Huotilainen, N. Ravaja, G. Jacucci. The Psychophysiology Primer: a guide to methods and a broad review with a focus on human-computer interaction. Foundations and Trends in Human-Computer Interaction, vol. 9, no. 3-4, pp. 150--307, 2016.