Managing Machine Learning Workflow Components

TL;DR

This paper introduces a machine learning workflow management approach using Hyperknowledge to improve the structuring, execution, and reuse of ML components, demonstrated through an Oil & Gas industry case study.

Contribution

It presents a novel ML workflow management technique leveraging Hyperknowledge for better component structuring, execution, and reuse, validated with a real-world case study.

Findings

Enhanced component retrieval and reuse in ML workflows

Effective structuring and execution of ML components using Hyperknowledge

Successful application in Oil & Gas industry case study

Abstract

Machine Learning Workflows (MLWfs) have become essential and a disruptive approach in problem-solving over several industries. However, the development process of MLWfs may be complicated, hard to achieve, time-consuming, and error-prone. To handle this problem, in this paper, we introduce machine learning workflow management (MLWfM) as a technique to aid the development and reuse of MLWfs and their components through three aspects: representation, execution, and creation. More precisely, we discuss our approach to structure the MLWfs' components and their metadata to aid retrieval and reuse of components in new MLWfs. Also, we consider the execution of these components within a tool. The hybrid knowledge representation, called Hyperknowledge, frames our methodology, supporting the three MLWfM's aspects. To validate our approach, we show a practical use case in the Oil & Gas industry.