Lost in Transmission: On the Impact of Networking Corruptions on Video Machine Learning Models

TL;DR

This paper investigates how networking-induced data corruptions impact video machine learning models, revealing artifacts, performance degradation, and the limited effectiveness of standard robustness methods across different tasks and datasets.

Contribution

It is the first comprehensive study on the effects of networking corruptions on video ML models, including analysis of artifacts, performance impacts, and evaluation of robustness strategies.

Findings

Networking corruptions cause visual and temporal artifacts.

Performance degradation varies by task and dataset.

Standard data augmentation does not fully recover performance.

Abstract

We study how networking corruptions--data corruptions caused by networking errors--affect video machine learning (ML) models. We discover apparent networking corruptions in Kinetics-400, a benchmark video ML dataset. In a simulation study, we investigate (1) what artifacts networking corruptions cause, (2) how such artifacts affect ML models, and (3) whether standard robustness methods can mitigate their negative effects. We find that networking corruptions cause visual and temporal artifacts (i.e., smeared colors or frame drops). These networking corruptions degrade performance on a variety of video ML tasks, but effects vary by task and dataset, depending on how much temporal context the tasks require. Lastly, we evaluate data augmentation--a standard defense for data corruptions--but find that it does not recover performance.