Demonstration of LogicLib: An Expressive Multi-Language Interface over Scalable Datalog System

TL;DR

LogicLib provides an expressive, multi-language interface over a scalable Datalog system, enabling complex logic-based algorithms to be easily integrated into Big Data analytics within the Apache Spark ecosystem.

Contribution

This paper introduces LogicLib, a library of recursive Datalog algorithms integrated with BigDatalog on Spark, offering a unified, user-friendly interface for big data logical queries.

Findings

Demonstrates how to express various applications using LogicLib.

Shows seamless integration with Spark DataFrame, MLlib, and Spark SQL.

Provides a user-friendly interface for interaction and monitoring.

Abstract

With the ever-increasing volume of data, there is an urgent need to provide expressive and efficient tools to support Big Data analytics. The declarative logical language Datalog has proven very effective at expressing concisely graph, machine learning, and knowledge discovery applications via recursive queries. In this demonstration, we develop Logic Library (LLib), a library of recursive algorithms written in Datalog that can be executed in BigDatalog, a Datalog engine on top of Apache Spark developed by us. LLib encapsulates complex logic-based algorithms into high-level APIs, which simplify the development and provide a unified interface akin to the one of Spark MLlib. As LLib is fully compatible with DataFrame, it enables the integrated utilization of its built-in applications and new Datalog queries with existing Spark functions, such as those provided by MLlib and Spark SQL. With a variety of examples, we will (i) show how to write programs with LLib to express a variety of applications; (ii) illustrate its user experience in Apache Spark ecosystem; and (iii) present a user-friendly interface to interact with the LLib framework and monitor the query results.