Implementing AI-powered semantic character recognition in motor racing sports

TL;DR

This paper introduces an AI-driven system that automates the recognition and tracking of drivers in motor racing broadcasts, enabling dynamic on-screen overlays without human input, demonstrated during live Formula E races.

Contribution

The paper presents a practical implementation of deep learning for real-time driver recognition and tracking in live motor racing broadcasts, improving efficiency and overlay accuracy.

Findings

System successfully deployed during live races

Enhanced overlay accuracy and real-time tracking

Reduced manual intervention in broadcast overlays

Abstract

Oftentimes TV producers of motor-racing programs overlay visual and textual media to provide on-screen context about drivers, such as a driver's name, position or photo. Typically this is accomplished by a human producer who visually identifies the drivers on screen, manually toggling the contextual media associated to each one and coordinating with cameramen and other TV producers to keep the racer in the shot while the contextual media is on screen. This labor-intensive and highly dedicated process is mostly suited to static overlays and makes it difficult to overlay contextual information about many drivers at the same time in short shots. This paper presents a system that largely automates these tasks and enables dynamic overlays using deep learning to track the drivers as they move on screen, without human intervention. This system is not merely theoretical, but an implementation has already been deployed during live races by a TV production company at Formula E races. We present the challenges faced during the implementation and discuss the implications. Additionally, we cover future applications and roadmap of this new technological development.