On Annotation-free Optimization of Video Coding for Machines

TL;DR

This paper introduces an annotation-free method for optimizing video coding for machines by measuring task loss directly on predictions, achieving rate savings up to 8.2% without ground truth data.

Contribution

It proposes a novel annotation-free optimization approach for video coding for machines that outperforms ground truth-based methods in rate savings.

Findings

Achieves up to 7.5% rate savings compared to traditional methods.

Further improves rate savings to 8.2% using non-annotated data.

Outperforms ground truth-based optimization in experimental results.

Abstract

Today, image and video data is not only viewed by humans, but also automatically analyzed by computer vision algorithms. However, current coding standards are optimized for human perception. Emerging from this, research on video coding for machines tries to develop coding methods designed for machines as information sink. Since many of these algorithms are based on neural networks, most proposals for video coding for machines build upon neural compression. So far, optimizing the compression by applying the task loss of the analysis network, for which ground truth data is needed, is achieving the best coding performance. But ground truth data is difficult to obtain and thus an optimization without ground truth is preferred. In this paper, we present an annotation-free optimization strategy for video coding for machines. We measure the distortion by calculating the task loss of the analysis network. Therefore, the predictions on the compressed image are compared with the predictions on the original image, instead of the ground truth data. Our results show that this strategy can even outperform training with ground truth data with rate savings of up to 7.5 %. By using the non-annotated training data, the rate gains can be further increased up to 8.2 %.