Towards Scalable Meta-Learning of near-optimal Interpretable Models via Synthetic Model Generations

TL;DR

This paper presents a scalable meta-learning approach for decision trees by synthetically generating near-optimal models, reducing computational costs while maintaining high performance in high-stakes applications.

Contribution

It introduces a novel synthetic data generation method for meta-learning decision trees, enabling scalable and efficient training without relying on real-world data.

Findings

Performance comparable to real-data pre-training

Significant reduction in computational costs

Enhanced flexibility in data generation

Abstract

Decision trees are widely used in high-stakes fields like finance and healthcare due to their interpretability. This work introduces an efficient, scalable method for generating synthetic pre-training data to enable meta-learning of decision trees. Our approach samples near-optimal decision trees synthetically, creating large-scale, realistic datasets. Using the MetaTree transformer architecture, we demonstrate that this method achieves performance comparable to pre-training on real-world data or with computationally expensive optimal decision trees. This strategy significantly reduces computational costs, enhances data generation flexibility, and paves the way for scalable and efficient meta-learning of interpretable decision tree models.