LightAutoML: AutoML Solution for a Large Financial Services Ecosystem

TL;DR

LightAutoML is a specialized AutoML system designed for a large financial ecosystem, achieving high-quality models faster than experienced data scientists and outperforming open-source AutoML tools in most cases.

Contribution

The paper introduces LightAutoML, a tailored AutoML framework optimized for complex financial ecosystems, demonstrating superior performance and efficiency over existing open-source solutions.

Findings

LightAutoML matches data scientists' performance in model quality.

It significantly reduces model development time.

It outperforms open-source AutoML tools on most benchmarks.

Abstract

We present an AutoML system called LightAutoML developed for a large European financial services company and its ecosystem satisfying the set of idiosyncratic requirements that this ecosystem has for AutoML solutions. Our framework was piloted and deployed in numerous applications and performed at the level of the experienced data scientists while building high-quality ML models significantly faster than these data scientists. We also compare the performance of our system with various general-purpose open source AutoML solutions and show that it performs better for most of the ecosystem and OpenML problems. We also present the lessons that we learned while developing the AutoML system and moving it into production.