Algorithmically generating new algebraic features of polynomial systems for machine learning

TL;DR

This paper introduces a framework for automatically generating algebraic features for machine learning models to optimize variable ordering in cylindrical algebraic decomposition, improving performance over human heuristics.

Contribution

It presents a systematic method to generate new features for ML models in polynomial system algorithms, specifically enhancing CAD variable ordering decisions.

Findings

ML-based variable ordering outperforms human heuristics

Additional features further improve ML decision accuracy

Framework can be adapted to other polynomial system algorithms

Abstract

There are a variety of choices to be made in both computer algebra systems (CASs) and satisfiability modulo theory (SMT) solvers which can impact performance without affecting mathematical correctness. Such choices are candidates for machine learning (ML) approaches, however, there are difficulties in applying standard ML techniques, such as the efficient identification of ML features from input data which is typically a polynomial system. Our focus is selecting the variable ordering for cylindrical algebraic decomposition (CAD), an important algorithm implemented in several CASs, and now also SMT-solvers. We created a framework to describe all the previously identified ML features for the problem and then enumerated all options in this framework to automatically generation many more features. We validate the usefulness of these with an experiment which shows that an ML choice for CAD variable ordering is superior to those made by human created heuristics, and further improved with these additional features. We expect that this technique of feature generation could be useful for other choices related to CAD, or even choices for other algorithms with polynomial systems for input.