SketchGraphNet: A Memory-Efficient Hybrid Graph Transformer for Large-Scale Sketch Corpora Recognition

TL;DR

This paper introduces SketchGraphNet, a memory-efficient hybrid graph transformer for large-scale sketch recognition, modeling sketches as graphs and achieving high accuracy with reduced memory and training time.

Contribution

It proposes a novel hybrid graph neural network architecture that combines local message passing with a memory-efficient global attention mechanism, and introduces a large-scale sketch benchmark dataset.

Findings

Achieves over 83% accuracy on large-scale sketch datasets.

Reduces GPU memory usage by over 40% compared to existing methods.

Maintains high accuracy while decreasing training time by more than 30%.

Abstract

This work investigates large-scale sketch recognition from a graph-native perspective, where free-hand sketches are directly modeled as structured graphs rather than raster images or stroke sequences. We propose SketchGraphNet, a hybrid graph neural architecture that integrates local message passing with a memory-efficient global attention mechanism, without relying on auxiliary positional or structural encodings. To support systematic evaluation, we construct SketchGraph, a large-scale benchmark comprising 3.44 million graph-structured sketches across 344 categories, with two variants (A and R) to reflect different noise conditions. Each sketch is represented as a spatiotemporal graph with normalized stroke-order attributes. On SketchGraph-A and SketchGraph-R, SketchGraphNet achieves Top-1 accuracies of 83.62% and 87.61%, respectively, under a unified training configuration. MemEffAttn further reduces peak GPU memory by over 40% and training time by more than 30% compared with Performer-based global attention, while maintaining comparable accuracy.