Prompting Neural-Guided Equation Discovery Based on Residuals

TL;DR

This paper introduces RED, a post-processing method that enhances neural-guided equation discovery by iteratively refining equations based on residuals, significantly improving the quality of discovered equations across various systems.

Contribution

RED provides a universal, fast, and extendable residual-based refinement technique for neural-guided and classical equation discovery systems.

Findings

RED improves all tested neural-guided systems

RED enhances classical genetic programming systems

Effective on 53 equations from the Feynman benchmark

Abstract

Neural-guided equation discovery systems use a data set as prompt and predict an equation that describes the data set without extensive search. However, if the equation does not meet the user's expectations, there are few options for getting other equation suggestions without intensive work with the system. To fill this gap, we propose Residuals for Equation Discovery (RED), a post-processing method that improves a given equation in a targeted manner, based on its residuals. By parsing the initial equation to a syntax tree, we can use node-based calculation rules to compute the residual for each subequation of the initial equation. It is then possible to use this residual as new target variable in the original data set and generate a new prompt. If, with the new prompt, the equation discovery system suggests a subequation better than the old subequation on a validation set, we replace the latter by the former. RED is usable with any equation discovery system, is fast to calculate, and is easy to extend for new mathematical operations. In experiments on 53 equations from the Feynman benchmark, we show that it not only helps to improve all tested neural-guided systems, but also all tested classical genetic programming systems.