Physically Embodied Deep Image Optimisation

TL;DR

This paper introduces a method that uses deep learning and differentiable rasterisation to optimize drawing strokes for robotic creation of physical sketches, enabling robots to replicate images with physical drawing instruments.

Contribution

It presents a novel approach combining deep networks and differentiable rasterisation for optimizing physical drawing primitives for robotic sketching.

Findings

Effective optimization of drawing strokes for physical robots

Ability to translate optimized strokes into G-code for robotic drawing

Demonstrated accurate reproduction of images by robots using the method

Abstract

Physical sketches are created by learning programs to control a drawing robot. A differentiable rasteriser is used to optimise sets of drawing strokes to match an input image, using deep networks to provide an encoding for which we can compute a loss. The optimised drawing primitives can then be translated into G-code commands which command a robot to draw the image using drawing instruments such as pens and pencils on a physical support medium.