Efficient Graph Compression Using Huffman Coding Based Techniques

TL;DR

This paper introduces Huffman coding inspired techniques to efficiently compress graph data, significantly reducing storage space and network transfer latency, facilitating large-scale graph analysis in resource-constrained environments.

Contribution

The paper presents novel graph compression methods based on pattern replacement with variable-length identifiers inspired by Huffman coding, achieving up to 80% space reduction.

Findings

Up to 80% reduction in graph storage space.

Applicable to various graph representations.

Improves data transfer efficiency in distributed systems.

Abstract

Graphs have been extensively used to represent data from various domains. In the era of Big Data, information is being generated at a fast pace, and analyzing the same is a challenge. Various methods have been proposed to speed up the analysis of the data and also mining it for information. All of this often involves using a massive array of compute nodes, and transmitting the data over the network. Of course, with the huge quantity of data, this poses a major issue to the task of gathering intelligence from data. Therefore, in order to address such issues with Big Data, using data compression techniques is a viable option. Since graphs represent most real world data, methods to compress graphs have been in the forefront of such endeavors. In this paper we propose techniques to compress graphs by finding specific patterns and replacing those with identifiers that are of variable length, an idea inspired by Huffman Coding. Specifically, given a graph G = (V, E), where V is the set of vertices and E is the set of edges, and |V| = n, we propose methods to reduce the space requirements of the graph by compressing the adjacency representation of the same. The proposed methods show up to 80% reduction is the space required to store the graphs as compared to using the adjacency matrix. The methods can also be applied to other representations as well. The proposed techniques help solve the issues related to computing on the graphs on resources limited compute nodes, as well as reduce the latency for transfer of data over the network in case of distributed computing.