How proofs are prepared at Camelot

TL;DR

This paper introduces a distributed verification framework based on Reed--Solomon encoding that enables robust, workload-balanced algorithms to count complex graph structures efficiently, matching the best sequential algorithms in total time.

Contribution

It presents a novel distributed verification framework that achieves resource efficiency and matches sequential algorithm performance for complex graph problems.

Findings

Counting and verifying k-cliques in graphs in near-optimal time and space.

Developing algorithms for counting triangles, chromatic polynomial, and Tutte polynomial.

Improved parallelizability and space efficiency over previous methods.

Abstract

We study a design framework for robust, independently verifiable, and workload-balanced distributed algorithms working on a common input. An algorithm based on the framework is essentially a distributed encoding procedure for a Reed--Solomon code, which enables (a) robustness against byzantine failures with intrinsic error-correction and identification of failed nodes, and (b) independent randomized verification to check the entire computation for correctness, which takes essentially no more resources than each node individually contributes to the computation. The framework builds on recent Merlin--Arthur proofs of batch evaluation of Williams~[{\em Electron.\ Colloq.\ Comput.\ Complexity}, Report TR16-002, January 2016] with the observation that {\em Merlin's magic is not needed} for batch evaluation---mere Knights can prepare the proof, in parallel, and with intrinsic error-correction. The contribution of this paper is to show that in many cases the verifiable batch evaluation framework admits algorithms that match in total resource consumption the best known sequential algorithm for solving the problem. As our main result, we show that the $k$-cliques in an $n$-vertex graph can be counted {\em and} verified in per-node $O(n^{(\omega+\epsilon)k/6})$ time and space on $O(n^{(\omega+\epsilon)k/6})$ compute nodes, for any constant $\epsilon>0$ and positive integer $k$ divisible by $6$, where $2\leq\omega<2.3728639$ is the exponent of matrix multiplication. This matches in total running time the best known sequential algorithm, due to Ne{\v{s}}et{\v{r}}il and Poljak [{\em Comment.~Math.~Univ.~Carolin.}~26 (1985) 415--419], and considerably improves its space usage and parallelizability. Further results include novel algorithms for counting triangles in sparse graphs, computing the chromatic polynomial of a graph, and computing the Tutte polynomial of a graph.