Proving Equivalence Between Imperative and MapReduce Implementations Using Program Transformations

TL;DR

This paper introduces a verification method that proves the correctness of MapReduce implementations by transforming imperative programs into equivalent MapReduce code, demonstrated through the PageRank algorithm.

Contribution

It presents a novel program transformation-based framework for verifying the equivalence between imperative and MapReduce implementations.

Findings

Successfully proved equivalence for PageRank implementation

Detailed eight-step transformation process explained

Framework can verify correctness of distributed algorithms

Abstract

Distributed programs are often formulated in popular functional frameworks like MapReduce, Spark and Thrill, but writing efficient algorithms for such frameworks is usually a non-trivial task. As the costs of running faulty algorithms at scale can be severe, it is highly desirable to verify their correctness. We propose to employ existing imperative reference implementations as specifications for MapReduce implementations. To this end, we present a novel verification approach in which equivalence between an imperative and a MapReduce implementation is established by a series of program transformations. In this paper, we present how the equivalence framework can be used to prove equivalence between an imperative implementation of the PageRank algorithm and its MapReduce variant. The eight individual transformation steps are individually presented and explained.