Video-based Sequential Bayesian Homography Estimation for Soccer Field Registration

TL;DR

This paper introduces a Bayesian framework for sequential homography estimation in soccer videos, improving accuracy and efficiency by modeling keypoint uncertainty and camera motion.

Contribution

It presents BHITK, a Bayesian homography inference method using Kalman filters, and refines datasets with a new annotation tool, advancing state-of-the-art in soccer field registration.

Findings

BHITK outperforms existing methods on homography metrics.

Refined datasets improve annotation accuracy.

Less computationally intensive methods achieve better results.

Abstract

A novel Bayesian framework is proposed, which explicitly relates the homography of one video frame to the next through an affine transformation while explicitly modelling keypoint uncertainty. The literature has previously used differential homography between subsequent frames, but not in a Bayesian setting. In cases where Bayesian methods have been applied, camera motion is not adequately modelled, and keypoints are treated as deterministic. The proposed method, Bayesian Homography Inference from Tracked Keypoints (BHITK), employs a two-stage Kalman filter and significantly improves existing methods. Existing keypoint detection methods may be easily augmented with BHITK. It enables less sophisticated and less computationally expensive methods to outperform the state-of-the-art approaches in most homography evaluation metrics. Furthermore, the homography annotations of the WorldCup and TS-WorldCup datasets have been refined using a custom homography annotation tool that has been released for public use. The refined datasets are consolidated and released as the consolidated and refined WorldCup (CARWC) dataset.