# Distributed Computation of Top-$k$ Degrees in Hidden Bipartite Graphs

## TL;DR

This paper introduces a distributed algorithm for efficiently identifying the top-$k$ high-degree nodes in hidden bipartite graphs using Apache Spark, reducing computational costs compared to centralized methods.

## Contribution

It presents a novel distributed approach for top-$k$ degree discovery in hidden bipartite graphs, leveraging Spark for improved performance.

## Key findings

- Significant performance improvements over centralized approaches.
- Effective reduction in the number of probing queries needed.
- Demonstrated scalability on large graphs.

## Abstract

Hidden graphs are flexible abstractions that are composed of a set of known vertices (nodes), whereas the set of edges are not known in advance. To uncover the set of edges, multiple edge probing queries must be executed by evaluating a function $f(u,v)$ that returns either true or false, if nodes $u$ and $v$ are connected or not respectively. Evidently, the graph can be revealed completely if all possible $n(n-1)/2$ probes are executed for a graph containing $n$ nodes. However, the function $f()$ is usually computationally intensive and therefore executing all possible probing queries result in high execution costs. The target is to provide answers to useful queries by executing as few probing queries as possible. In this work, we study the problem of discovering the top-$k$ nodes of a hidden bipartite graph with the highest degrees, by using distributed algorithms. In particular, we use Apache Spark and provide experimental results showing that significant performance improvements are achieved in comparison to existing centralized approaches.