# Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography

## TL;DR

This paper presents a deep reinforcement learning approach for autonomous aerial cinematography, enabling drones to learn aesthetic shot selection from experience and generalize from simulation to real-world filming.

## Contribution

It introduces a novel learning scheme that uses aesthetic features to train a drone to select desirable shots without human guidance, demonstrating successful transfer from simulation to real drone.

## Key findings

- The drone learned to select aesthetically pleasing shots in simulation.
- The approach generalizes well to real-world drone filming.
- User study shows high satisfaction with the shot quality.

## Abstract

Aerial filming is constantly gaining importance due to the recent advances in drone technology. It invites many intriguing, unsolved problems at the intersection of aesthetical and scientific challenges. In this work, we propose a deep reinforcement learning agent which supervises motion planning of a filming drone by making desirable shot mode selections based on aesthetical values of video shots. Unlike most of the current state-of-the-art approaches that require explicit guidance by a human expert, our drone learns how to make favorable viewpoint selections by experience. We propose a learning scheme that exploits aesthetical features of retrospective shots in order to extract a desirable policy for better prospective shots. We train our agent in realistic AirSim simulations using both a hand-crafted reward function as well as reward from direct human input. We then deploy the same agent on a real DJI M210 drone in order to test the generalization capability of our approach to real world conditions. To evaluate the success of our approach in the end, we conduct a comprehensive user study in which participants rate the shot quality of our methods. Videos of the system in action can be seen at https://youtu.be/qmVw6mfyEmw.