HAVANA: Hierarchical stochastic neighbor embedding for Accelerated Video ANnotAtions

TL;DR

This paper introduces HAVANA, a hierarchical embedding method that significantly accelerates video annotation by reducing manual effort through multi-scale feature exploration, demonstrating over 10x efficiency gains.

Contribution

The paper proposes a novel hierarchical stochastic neighbor embedding technique for efficient, scalable video annotation, improving annotation speed and robustness across datasets.

Findings

Achieved over 10x reduction in annotation clicks for 12 hours of video.

Demonstrated robustness of the pipeline across multiple datasets.

Identified optimal HSNE parameters for different scenarios.

Abstract

Video annotation is a critical and time-consuming task in computer vision research and applications. This paper presents a novel annotation pipeline that uses pre-extracted features and dimensionality reduction to accelerate the temporal video annotation process. Our approach uses Hierarchical Stochastic Neighbor Embedding (HSNE) to create a multi-scale representation of video features, allowing annotators to efficiently explore and label large video datasets. We demonstrate significant improvements in annotation effort compared to traditional linear methods, achieving more than a 10x reduction in clicks required for annotating over 12 hours of video. Our experiments on multiple datasets show the effectiveness and robustness of our pipeline across various scenarios. Moreover, we investigate the optimal configuration of HSNE parameters for different datasets. Our work provides a promising direction for scaling up video annotation efforts in the era of video understanding.