SLEGO: A Collaborative Data Analytics System with LLM Recommender for Diverse Users

TL;DR

SLEGO is a collaborative, cloud-based data analytics platform that uses microservices, a knowledge base, and an LLM recommender to enable users of varying expertise to build analytics pipelines efficiently.

Contribution

The paper introduces SLEGO, a modular analytics system that integrates LLM-based recommendations and knowledge sharing to democratize data analytics for diverse users.

Findings

Enhanced microservice reuse in analytics pipelines

Improved collaboration between developers and novices

Effective LLM-based recommendations for tool selection

Abstract

This paper presents the SLEGO (Software-Lego) system, a collaborative analytics platform that bridges the gap between experienced developers and novice users using a cloud-based platform with modular, reusable microservices. These microservices enable developers to share their analytical tools and workflows, while a simple graphical user interface (GUI) allows novice users to build comprehensive analytics pipelines without programming skills. Supported by a knowledge base and a Large Language Model (LLM) powered recommendation system, SLEGO enhances the selection and integration of microservices, increasing the efficiency of analytics pipeline construction. Case studies in finance and machine learning illustrate how SLEGO promotes the sharing and assembly of modular microservices, significantly improving resource reusability and team collaboration. The results highlight SLEGO's role in democratizing data analytics by integrating modular design, knowledge bases, and recommendation systems, fostering a more inclusive and efficient analytical environment.