Image-driven Robot Drawing with Rapid Lognormal Movements

TL;DR

This paper introduces a method for generating human-like robot drawing motions guided by image objectives, using a sigma-lognormal movement model integrated with differentiable rendering for realistic and feasible trajectories.

Contribution

It presents a novel integration of the sigma-lognormal movement model with differentiable rendering to produce natural, image-guided robot drawing motions.

Findings

Generated realistic, human-like robot drawing trajectories.

Successfully reproduced synthetic graffiti and image abstraction.

Enhanced visual aesthetics and naturalness of robot drawings.

Abstract

Large image generation and vision models, combined with differentiable rendering technologies, have become powerful tools for generating paths that can be drawn or painted by a robot. However, these tools often overlook the intrinsic physicality of the human drawing/writing act, which is usually executed with skillful hand/arm gestures. Taking this into account is important for the visual aesthetics of the results and for the development of closer and more intuitive artist-robot collaboration scenarios. We present a method that bridges this gap by enabling gradient-based optimization of natural human-like motions guided by cost functions defined in image space. To this end, we use the sigma-lognormal model of human hand/arm movements, with an adaptation that enables its use in conjunction with a differentiable vector graphics (DiffVG) renderer. We demonstrate how this pipeline can be used to generate feasible trajectories for a robot by combining image-driven objectives with a minimum-time smoothing criterion. We demonstrate applications with generation and robotic reproduction of synthetic graffiti as well as image abstraction.