ViFusion: In-Network Tensor Fusion for Scalable Video Feature Indexing

TL;DR

ViFusion is a novel in-network tensor fusion framework that significantly enhances the scalability and efficiency of large-scale video feature indexing in datacenter environments, achieving up to 22 times throughput improvement.

Contribution

The paper introduces ViFusion, a new communication-aware tensor fusion method that integrates in-network computation to optimize distributed video indexing workflows.

Findings

Achieves 8-22x throughput improvement in video retrieval systems.

Effectively consolidates small feature tensors to reduce data transfer overhead.

Demonstrates robustness across heterogeneous hardware and complex topologies.

Abstract

Large-scale video feature indexing in datacenters is critically dependent on efficient data transfer. Although in-network computation has emerged as a compelling strategy for accelerating feature extraction and reducing overhead in distributed multimedia systems, harnessing advanced networking resources at both the switch and host levels remains a formidable challenge. These difficulties are compounded by heterogeneous hardware, diverse application requirements, and complex multipath topologies. Existing methods focus primarily on optimizing inference for large neural network models using specialized collective communication libraries, which often face performance degradation in network congestion scenarios. To overcome these limitations, we present ViFusion, a communication aware tensor fusion framework that streamlines distributed video indexing by merging numerous small feature tensors into consolidated and more manageable units. By integrating an in-network computation module and a dedicated tensor fusion mechanism within datacenter environments, ViFusion substantially improves the efficiency of video feature indexing workflows. The deployment results show that ViFusion improves the throughput of the video retrieval system by 8--22 times with the same level of latency as state-of-the-art systems.