# MPP: Model Performance Predictor

## TL;DR

This paper introduces MPP, an ensemble-based machine learning approach to predict model performance in production environments where true labels are unavailable, aiding automated monitoring and management of deployed models.

## Contribution

The paper presents a novel ensemble-based predictor for model performance in production, enabling automated monitoring without access to true labels.

## Key findings

- MPP effectively predicts model performance in real-time.
- The ensemble score correlates well with actual performance metrics.
- Automates alerting and management of ML deployments at scale.

## Abstract

Operations is a key challenge in the domain of machine learning pipeline deployments involving monitoring and management of real-time prediction quality. Typically, metrics like accuracy, RMSE etc., are used to track the performance of models in deployment. However, these metrics cannot be calculated in production due to the absence of labels. We propose using an ML algorithm, Model Performance Predictor (MPP), to track the performance of the models in deployment. We argue that an ensemble of such metrics can be used to create a score representing the prediction quality in production. This in turn facilitates formulation and customization of ML alerts, that can be escalated by an operations team to the data science team. Such a score automates monitoring and enables ML deployments at scale.