MambaPainter: Neural Stroke-Based Rendering in a Single Step

TL;DR

MambaPainter is a fast neural stroke-based rendering method that predicts over 100 brush strokes in a single step, enabling efficient high-quality oil painting-style image translation.

Contribution

It introduces a novel single-step sequence prediction approach using the selective state-space model for rapid stroke-based rendering.

Findings

Predicts over 100 brush strokes in one inference

Achieves high-quality oil painting-style images efficiently

Outperforms state-of-the-art methods in speed and quality

Abstract

Stroke-based rendering aims to reconstruct an input image into an oil painting style by predicting brush stroke sequences. Conventional methods perform this prediction stroke-by-stroke or require multiple inference steps due to the limitations of a predictable number of strokes. This procedure leads to inefficient translation speed, limiting their practicality. In this study, we propose MambaPainter, capable of predicting a sequence of over 100 brush strokes in a single inference step, resulting in rapid translation. We achieve this sequence prediction by incorporating the selective state-space model. Additionally, we introduce a simple extension to patch-based rendering, which we use to translate high-resolution images, improving the visual quality with a minimal increase in computational cost. Experimental results demonstrate that MambaPainter can efficiently translate inputs to oil painting-style images compared to state-of-the-art methods. The codes are available at https://github.com/STomoya/MambaPainter.