On the Temporality for Sketch Representation Learning

TL;DR

This paper investigates the role of temporal information in sketch representation learning, finding that absolute coordinates and non-autoregressive decoders are more effective, with the importance of temporality varying by task and order.

Contribution

It provides a comprehensive analysis of temporal aspects in sketch sequences, highlighting the effectiveness of absolute coordinates and non-autoregressive models over traditional methods.

Findings

Absolute coordinates outperform relative ones in sketch modeling.

Non-autoregressive decoders outperform autoregressive models.

The importance of temporality varies depending on the task and sequence order.

Abstract

Sketches are simple human hand-drawn abstractions of complex scenes and real-world objects. Although the field of sketch representation learning has advanced significantly, there is still a gap in understanding the true relevance of the temporal aspect to the quality of these representations. This work investigates whether it is indeed justifiable to treat sketches as sequences, as well as which internal orders play a more relevant role. The results indicate that, although the use of traditional positional encodings is valid for modeling sketches as sequences, absolute coordinates consistently outperform relative ones. Furthermore, non-autoregressive decoders outperform their autoregressive counterparts. Finally, the importance of temporality was shown to depend on both the order considered and the task evaluated.