# A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning

## TL;DR

This paper introduces a robust ensemble method based on the median-of-means principle for hyperparameter tuning and model selection, effectively handling outliers and heavy-tailed data in automated machine learning.

## Contribution

It proposes a novel robust selection procedure and ensemble method that improve hyperparameter tuning and model selection in the presence of corrupted data.

## Key findings

- Effective in selecting algorithms with heavy-tailed data
- Automatically tunes hyperparameters robustly
- Scalable for autoML applications

## Abstract

Hyperparameters tuning and model selection are important steps in machine learning. Unfortunately, classical hyperparameter calibration and model selection procedures are sensitive to outliers and heavy-tailed data. In this work, we construct a selection procedure which can be seen as a robust alternative to cross-validation and is based on a median-of-means principle. Using this procedure, we also build an ensemble method which, trained with algorithms and corrupted heavy-tailed data, selects an algorithm, trains it with a large uncorrupted subsample and automatically tune its hyperparameters. The construction relies on a divide-and-conquer methodology, making this method easily scalable for autoML given a corrupted database. This method is tested with the LASSO which is known to be highly sensitive to outliers.