# ArtifactOps and ArtifactDL: a methodology and a language for conceptualizing and operationalising different types of pipelines

## TL;DR

This paper introduces ArtifactOps and ArtifactDL, a new methodology and language for designing and managing complex machine learning pipelines beyond simple model deployment.

## Contribution

The novelty lies in proposing a unified methodology (ArtifactOps) and a pipeline definition language (ArtifactDL) that extend beyond traditional MLOps to handle diverse pipeline types.

## Key findings

- ArtifactOps unifies XXOps paradigms by addressing shared stages across different pipeline contexts.
- ArtifactDL provides a structured way to define pipeline aspects, supporting the ArtifactOps methodology.
- Expert evaluation and real-world scenarios validate the effectiveness of the proposed approach.

## Abstract

Machine learning is already integrated in diverse domains enhancing their performance and decision support. For laboratories, this approach is normally sufficient. However, in real environments, these models can not be generally deployed isolated since they require additional steps to satisfy an objective. These steps can range from different data transformations to the inclusion of extra machine learning models which compose an analytic pipeline. Moreover, the majority of software solutions wrap a model into an API and, rarely, focus on the whole pipeline. These are unresolved topics in the well-known MLOps methodology, specifically in packaging and service phases. In addition, these concerns can also be extrapolated to other paradigms like DevOps or DataOps.

In the context of the Pliades European project, this paper approaches the conceptualization of diverse types of pipelines from different perspectives and for different contexts, instead of simplifying the deployment and serving to an API.

Thus, ArtifactOps methodology is proposed aimed at unifying XXOps paradigms which share the majority of stages. Finally, ArtifactDL pipeline definition language is proposed to describe the key aspects identified when designing different pipelines types and to support the proposed ArtifactOps methodology. Moreover, the research presents two real scenarios to better illustrate both ArtifactOps methodology and ArtifactDL pipeline definition language and it is defined an expert evaluation conducted to validate the approach.