Cloud-native RStudio on Kubernetes for Hopsworks

TL;DR

This paper presents a cloud-native, multi-tenant RStudio server system on Kubernetes integrated with Hopsworks, enabling secure, scalable, and customizable RStudio instances for data science in a cloud environment.

Contribution

It introduces a novel multi-tenant RStudio deployment on Kubernetes within Hopsworks, addressing security, performance isolation, and scalability challenges in cloud environments.

Findings

Supports 44 concurrent RStudio servers on a small cluster

System can scale to hundreds of servers with additional resources

Ensures secure data sharing and user customization

Abstract

In order to fully benefit from cloud computing, services are designed following the "multi-tenant" architectural model, which is aimed at maximizing resource sharing among users. However, multi-tenancy introduces challenges of security, performance isolation, scaling, and customization. RStudio server is an open-source Integrated Development Environment (IDE) accessible over a web browser for the R programming language. We present the design and implementation of a multi-user distributed system on Hopsworks, a data-intensive AI platform, following the multi-tenant model that provides RStudio as Software as a Service (SaaS). We use the most popular cloud-native technologies: Docker and Kubernetes, to solve the problems of performance isolation, security, and scaling that are present in a multi-tenant environment. We further enable secure data sharing in RStudio server instances to provide data privacy and allow collaboration among RStudio users. We integrate our system with Apache Spark, which can scale and handle Big Data processing workloads. Also, we provide a UI where users can provide custom configurations and have full control of their own RStudio server instances. Our system was tested on a Google Cloud Platform cluster with four worker nodes, each with 30GB of RAM allocated to them. The tests on this cluster showed that 44 RStudio servers, each with 2GB of RAM, can be run concurrently. Our system can scale out to potentially support hundreds of concurrently running RStudio servers by adding more resources (CPUs and RAM) to the cluster or system.