StrokeNet: Unveiling How to Learn Fine-Grained Interactions in Online Handwritten Stroke Classification

TL;DR

StrokeNet introduces a novel architecture that models fine-grained stroke interactions in online handwritten Chinese character recognition by using reference points and specialized attention mechanisms, significantly improving accuracy.

Contribution

The paper proposes StrokeNet, a new network that encodes strokes with reference points and features, employing dynamic selection and attention modules to better capture local interactions.

Findings

Achieves state-of-the-art accuracy on multiple datasets.

Improves CASIA-onDo from 93.81% to 95.54%.

Demonstrates robustness and effectiveness of the approach.

Abstract

Stroke classification remains challenging due to variations in writing style, ambiguous content, and dynamic writing positions. The core challenge in stroke classification is modeling the semantic relationships between strokes. Our observations indicate that stroke interactions are typically localized, making it difficult for existing deep learning methods to capture such fine-grained relationships. Although viewing strokes from a point-level perspective can address this issue, it introduces redundancy. However, by selecting reference points and using their sequential order to represent strokes in a fine-grained manner, this problem can be effectively solved. This insight inspired StrokeNet, a novel network architecture encoding strokes as reference pair representations (points + feature vectors), where reference points enable spatial queries and features mediate interaction modeling. Specifically, we dynamically select reference points for each stroke and sequence them, employing an Inline Sequence Attention (ISA) module to construct contextual features. To capture spatial feature interactions, we devised a Cross-Ellipse Query (CEQ) mechanism that clusters reference points and extracts features across varying spatial scales. Finally, a joint optimization framework simultaneously predicts stroke categories via reference points regression and adjacent stroke semantic transition modeling through an Auxiliary Branch (Aux-Branch). Experimental results show that our method achieves state-of-the-art performance on multiple public online handwritten datasets. Notably, on the CASIA-onDo dataset, the accuracy improves from 93.81$\%$ to 95.54$\%$, demonstrating the effectiveness and robustness of our approach.