MLOps Challenges in Multi-Organization Setup: Experiences from Two Real-World Cases

TL;DR

This paper explores the challenges of implementing MLOps in multi-organization environments through two real-world case studies, highlighting integration issues, scaling, and deployment complexities.

Contribution

It provides practical insights into MLOps challenges in multi-organization settings and proposes future research directions based on real-world experiences.

Findings

Integration between organizations is complex and requires specialized mechanisms.

Scaling MLOps across multiple organizations introduces unique challenges.

Continuous deployment practices need adaptation for multi-organization AI/ML systems.

Abstract

The emerging age of connected, digital world means that there are tons of data, distributed to various organizations and their databases. Since this data can be confidential in nature, it cannot always be openly shared in seek of artificial intelligence (AI) and machine learning (ML) solutions. Instead, we need integration mechanisms, analogous to integration patterns in information systems, to create multi-organization AI/ML systems. In this paper, we present two real-world cases. First, we study integration between two organizations in detail. Second, we address scaling of AI/ML to multi-organization context. The setup we assume is that of continuous deployment, often referred to DevOps in software development. When also ML components are deployed in a similar fashion, term MLOps is used. Towards the end of the paper, we list the main observations and draw some final conclusions. Finally, we propose some directions for future work.