TRACE: A Differentiable Approach to Line-level Stroke Recovery for Offline Handwritten Text

TL;DR

TRACE is a novel differentiable model that recovers stroke order and velocity from offline handwritten text, enabling improved handwriting synthesis and establishing new benchmarks for stroke recovery.

Contribution

It introduces the first end-to-end trainable system for line-level stroke recovery from offline text without pre/post-processing, using a CRNN and DTW for alignment.

Findings

Recovered stroke trajectories improve handwriting synthesis.

Established first benchmarks for offline stroke recovery.

Demonstrated end-to-end training on arbitrary line widths.

Abstract

Stroke order and velocity are helpful features in the fields of signature verification, handwriting recognition, and handwriting synthesis. Recovering these features from offline handwritten text is a challenging and well-studied problem. We propose a new model called TRACE (Trajectory Recovery by an Adaptively-trained Convolutional Encoder). TRACE is a differentiable approach that uses a convolutional recurrent neural network (CRNN) to infer temporal stroke information from long lines of offline handwritten text with many characters and dynamic time warping (DTW) to align predictions and ground truth points. TRACE is perhaps the first system to be trained end-to-end on entire lines of text of arbitrary width and does not require the use of dynamic exemplars. Moreover, the system does not require images to undergo any pre-processing, nor do the predictions require any post-processing. Consequently, the recovered trajectory is differentiable and can be used as a loss function for other tasks, including synthesizing offline handwritten text. We demonstrate that temporal stroke information recovered by TRACE from offline data can be used for handwriting synthesis and establish the first benchmarks for a stroke trajectory recovery system trained on the IAM online handwriting dataset.