A Deformable Interface for Human Touch Recognition using Stretchable Carbon Nanotube Dielectric Elastomer Sensors and Deep Neural Networks

TL;DR

This paper introduces OrbTouch, a soft, deformable interface that uses stretchable sensors and deep learning to recognize human touch gestures and locations, enabling new human-computer interaction methods.

Contribution

The work presents a novel soft interface with integrated stretchable sensors and deep neural networks for accurate touch and gesture recognition, advancing human-computer interaction technology.

Findings

Successfully classified gestures and touch locations using CNNs.

Demonstrated control of a game (Tetris) via soft interface.

Provided a modular framework for interpreting deformation signals.

Abstract

User interfaces provide an interactive window between physical and virtual environments. A new concept in the field of human-computer interaction is a soft user interface; a compliant surface that facilitates touch interaction through deformation. Despite the potential of these interfaces, they currently lack a signal processing framework that can efficiently extract information from their deformation. Here we present OrbTouch, a device that uses statistical learning algorithms, based on convolutional neural networks, to map deformations from human touch to categorical labels (i.e., gestures) and touch location using stretchable capacitor signals as inputs. We demonstrate this approach by using the device to control the popular game Tetris. OrbTouch provides a modular, robust framework to interpret deformation in soft media, laying a foundation for new modes of human computer interaction through shape changing solids.

Full text

See pages 1-last of A_Deformable_Interface_for_Human_Touch_Recognition_using_Stretchable_Carbon_Nanotube_Dielectric_Elastomer_Sensors_and_Deep_Neural_Networks_arxiv.pdf