Sensory Optimization: Neural Networks as a Model for Understanding and Creating Art

TL;DR

This paper explores how neural networks can model human art understanding and creation by optimizing visual features to evoke object recognition, shedding light on cognitive and cultural aspects of art.

Contribution

It demonstrates that CNNs can generate and interpret stylized art through optimization, proposing a model for human artistic processes based on visual effects on recognition systems.

Findings

CNNs can recognize objects without training on art.

Deep Dream and Style Transfer produce basic visual art forms.

Artists may optimize art for effects on human recognition systems.

Abstract

This article is about the cognitive science of visual art. Artists create physical artifacts (such as sculptures or paintings) which depict people, objects, and events. These depictions are usually stylized rather than photo-realistic. How is it that humans are able to understand and create stylized representations? Does this ability depend on general cognitive capacities or an evolutionary adaptation for art? What role is played by learning and culture? Machine Learning can shed light on these questions. It's possible to train convolutional neural networks (CNNs) to recognize objects without training them on any visual art. If such CNNs can generalize to visual art (by creating and understanding stylized representations), then CNNs provide a model for how humans could understand art without innate adaptations or cultural learning. I argue that Deep Dream and Style Transfer show that CNNs can create a basic form of visual art, and that humans could create art by similar processes. This suggests that artists make art by optimizing for effects on the human object-recognition system. Physical artifacts are optimized to evoke real-world objects for this system (e.g. to evoke people or landscapes) and to serve as superstimuli for this system.