Chameleon: A Semi-AutoML framework targeting quick and scalable development and deployment of production-ready ML systems for SMEs

TL;DR

Chameleon is a semi-AutoML framework designed to enable small and medium-sized enterprises to quickly develop, scale, and deploy production-ready machine learning systems tailored for real-world data challenges.

Contribution

It introduces a semi-AutoML framework with RWD-relevant defaults, automation tools, and testing components to streamline ML development and deployment for SMEs.

Findings

Effective handling of real-world data challenges for SMEs

Automated experiment iteration improves development speed

Testing framework supports deployment governance

Abstract

Developing, scaling, and deploying modern Machine Learning solutions remains challenging for small- and middle-sized enterprises (SMEs). This is due to a high entry barrier of building and maintaining a dedicated IT team as well as the difficulties of real-world data (RWD) compared to standard benchmark data. To address this challenge, we discuss the implementation and concepts of Chameleon, a semi-AutoML framework. The goal of Chameleon is fast and scalable development and deployment of production-ready machine learning systems into the workflow of SMEs. We first discuss the RWD challenges faced by SMEs. After, we outline the central part of the framework which is a model and loss-function zoo with RWD-relevant defaults. Subsequently, we present how one can use a templatable framework in order to automate the experiment iteration cycle, as well as close the gap between development and deployment. Finally, we touch on our testing framework component allowing us to investigate common model failure modes and support best practices of model deployment governance.