Handwritten Character Recognition from Wearable Passive RFID

TL;DR

This paper presents a novel wearable RFID-based system for recognizing handwritten characters by combining sequence and bitmap data, using a specialized CNN architecture, achieving 72% accuracy on a new dataset.

Contribution

It introduces a new wearable electro-textile sensor, a data fusion preprocessing pipeline, and a CNN with a novel upsampling structure for small input size recognition.

Findings

Achieved 72% accuracy on the dataset

Collected 7500 characters from 10 subjects

Released data and model publicly

Abstract

In this paper we study the recognition of handwritten characters from data captured by a novel wearable electro-textile sensor panel. The data is collected sequentially, such that we record both the stroke order and the resulting bitmap. We propose a preprocessing pipeline that fuses the sequence and bitmap representations together. The data is collected from ten subjects containing altogether 7500 characters. We also propose a convolutional neural network architecture, whose novel upsampling structure enables successful use of conventional ImageNet pretrained networks, despite the small input size of only 10x10 pixels. The proposed model reaches 72\% accuracy in experimental tests, which can be considered good accuracy for this challenging dataset. Both the data and the model are released to the public.