SketchLattice: Latticed Representation for Sketch Manipulation

TL;DR

This paper introduces SketchLattice, a lattice-based sketch representation that captures structural cues without needing vector data, enabling efficient sketch manipulation tasks like healing and synthesis.

Contribution

We propose a novel lattice structured sketch representation that preserves structural cues and reduces model complexity, improving sketch manipulation capabilities.

Findings

Lattice structure effectively encodes sketch details.

Our model uses 13.5 times fewer parameters than state-of-the-art.

SketchLattice improves sketch healing and image-to-sketch synthesis.

Abstract

The key challenge in designing a sketch representation lies with handling the abstract and iconic nature of sketches. Existing work predominantly utilizes either, (i) a pixelative format that treats sketches as natural images employing off-the-shelf CNN-based networks, or (ii) an elaborately designed vector format that leverages the structural information of drawing orders using sequential RNN-based methods. While the pixelative format lacks intuitive exploitation of structural cues, sketches in vector format are absent in most cases limiting their practical usage. Hence, in this paper, we propose a lattice structured sketch representation that not only removes the bottleneck of requiring vector data but also preserves the structural cues that vector data provides. Essentially, sketch lattice is a set of points sampled from the pixelative format of the sketch using a lattice graph. We show that our lattice structure is particularly amenable to structural changes that largely benefits sketch abstraction modeling for generation tasks. Our lattice representation could be effectively encoded using a graph model, that uses significantly fewer model parameters (13.5 times lesser) than existing state-of-the-art. Extensive experiments demonstrate the effectiveness of sketch lattice for sketch manipulation, including sketch healing and image-to-sketch synthesis.