Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach

TL;DR

This paper introduces a framework that converts classical ML pipelines into neural networks, enabling joint training via backpropagation to improve overall accuracy.

Contribution

It proposes a novel translation approach that allows classical ML pipelines to be fine-tuned end-to-end as neural networks, enhancing their performance.

Findings

Fine-tuning translated pipelines increases accuracy.

The approach enables joint optimization of pipeline components.

Experimental results demonstrate improved performance.

Abstract

Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML pipelines is sub-optimal, since it cannot jointly optimize multiple components. To this end, we propose a framework that translates a pre-trained ML pipeline into a neural network and fine-tunes the ML models within the pipeline jointly using backpropagation. Our experiments show that fine-tuning of the translated pipelines is a promising technique able to increase the final accuracy.