Opt-In Art: Learning Art Styles Only from Few Examples

TL;DR

This paper demonstrates that a text-to-image model trained solely on photographs can be adapted to generate artistic styles with few examples, eliminating the need for pre-training on artistic datasets.

Contribution

It introduces a method for learning artistic styles from minimal examples without prior artistic data exposure, challenging the necessity of extensive artistic pre-training.

Findings

Models can adapt to artistic styles with few examples

High-quality artistic images can be generated without prior artistic data

Analysis shows non-artistic data contributes to artistic style generation

Abstract

We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investigate this, we train a text-to-image model exclusively on photographs, without access to any painting-related content. We show that it is possible to adapt a model that is trained without paintings to an artistic style, given only few examples. User studies and automatic evaluations confirm that our model (post-adaptation) performs on par with state-of-the-art models trained on massive datasets that contain artistic content like paintings, drawings or illustrations. Finally, using data attribution techniques, we analyze how both artistic and non-artistic datasets contribute to generating artistic-style images. Surprisingly, our findings suggest that high-quality artistic outputs can be achieved without prior exposure to artistic data, indicating that artistic style generation can occur in a controlled, opt-in manner using only a limited, carefully selected set of training examples.