GO-GenZip: Goal-Oriented Generative Sampling and Hybrid Compression

TL;DR

This paper introduces GO-GenZip, a goal-oriented generative sampling and hybrid compression framework that reduces network telemetry data volume by over 50% while preserving analytical fidelity, through adaptive sampling and AI-driven encoding.

Contribution

It presents a novel framework combining adaptive sampling and generative AI-based compression, optimizing data collection and encoding for network telemetry.

Findings

Achieves over 50% reduction in data transfer costs.

Maintains comparable accuracy in downstream analysis.

Demonstrates effectiveness on real network datasets.

Abstract

Current network data telemetry pipelines consist of massive streams of fine-grained Key Performance Indicators (KPIs) from multiple distributed sources towards central aggregators, making data storage, transmission, and real-time analysis increasingly unsustainable. This work presents a generative AI (GenAI)-driven sampling and hybrid compression framework that redesigns network telemetry from a goal-oriented perspective. Unlike conventional approaches that passively compress fully observed data, our approach jointly optimizes what to observe and how to encode it, guided by the relevance of information to downstream tasks. The framework integrates adaptive sampling policies, using adaptive masking techniques, with generative modeling to identify patterns and preserve critical features across temporal and spatial dimensions. The selectively acquired data are further processed through a hybrid compression scheme that combines traditional lossless coding with GenAI-driven, lossy compression. Experimental results on real network datasets demonstrate over 50$\%$ reductions in sampling and data transfer costs, while maintaining comparable reconstruction accuracy and goal-oriented analytical fidelity in downstream tasks.