# A Robust Billboard-based Free-viewpoint Video Synthesizing Algorithm for Sports Scenes

## TL;DR

This paper introduces a robust billboard-based algorithm for free-viewpoint video synthesis in sports scenes, effectively handling occlusions and working under wide-baseline camera setups without requiring high-precision 3D models.

## Contribution

It proposes a novel method that uses raw space carving models for robust object segmentation and occlusion detection in free-viewpoint video synthesis.

## Key findings

- Successfully synthesized free-viewpoint videos of soccer scenes
- Outperformed state-of-the-art methods in handling occlusions
- Effective in wide-baseline camera configurations

## Abstract

We present a billboard-based free-viewpoint video synthesizing algorithm for sports scenes that can robustly reconstruct and render a high-fidelity billboard model for each object, including an occluded one, in each camera. Its contributions are (1) applicable to a challenging shooting condition where a high precision 3D model cannot be built because a small number of cameras featuring wide-baseline are equipped; (2) capable of reproducing appearances of occlusions, that is one of the most significant issues for billboard-based approaches due to the ineffective detection of overlaps. To achieve contributions above, the proposed method does not attempt to find a high-quality 3D model but utilizes a raw 3D model that is obtained directly from space carving. Although the model is insufficiently accurate for producing an impressive visual effect, precise objects segmentation and occlusions detection can be performed by back-projecting it onto each camera plane. The billboard model of each object in each camera is rendered according to whether it is occluded or not, and its location in the virtual stadium is determined considering the location of its 3D model. We synthesized free-viewpoint videos of two soccer sequences recorded by five cameras with the proposed and state-of-art methods to demonstrate its performance.