Sports Camera Calibration via Synthetic Data

TL;DR

This paper introduces a fully automatic sports camera calibration method using synthetic data, deep learning, and GANs, achieving state-of-the-art accuracy on soccer and volleyball datasets.

Contribution

It presents a novel camera pose engine, a deep feature learning approach, and a two-GAN model for robust calibration from a single image.

Findings

Robust calibration on synthetic data

State-of-the-art accuracy on soccer dataset

High performance on volleyball dataset

Abstract

Calibrating sports cameras is important for autonomous broadcasting and sports analysis. Here we propose a highly automatic method for calibrating sports cameras from a single image using synthetic data. First, we develop a novel camera pose engine. The camera pose engine has only three significant free parameters so that it can effectively generate a lot of camera poses and corresponding edge (i.e, field marking) images. Then, we learn compact deep features via a siamese network from paired edge image and camera pose and build a feature-pose database. After that, we use a novel two-GAN (generative adversarial network) model to detect field markings in real images. Finally, we query an initial camera pose from the feature-pose database and refine camera poses using truncated distance images. We evaluate our method on both synthetic and real data. Our method not only demonstrates the robustness on the synthetic data but also achieves the state-of-the-art accuracy on a standard soccer dataset and very high performance on a volleyball dataset.