CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad

TL;DR

CausalEvolve introduces a causal scratchpad mechanism for LLM-based evolutionary agents, enhancing guidance, knowledge organization, and efficiency in solving complex scientific problems.

Contribution

It presents a novel causal scratchpad that enables LLM agents to identify, reason about, and hypothesize guiding factors, improving open-ended scientific discovery.

Findings

CausalEvolve improves evolutionary efficiency in scientific tasks.

It discovers better solutions compared to previous methods.

The causal scratchpad effectively guides the evolution process.

Abstract

Evolve-based agent such as AlphaEvolve is one of the notable successes in using Large Language Models (LLMs) to build AI Scientists. These agents tackle open-ended scientific problems by iteratively improving and evolving programs, leveraging the prior knowledge and reasoning capabilities of LLMs. Despite the success, existing evolve-based agents lack targeted guidance for evolution and effective mechanisms for organizing and utilizing knowledge acquired from past evolutionary experience. Consequently, they suffer from decreasing evolution efficiency and exhibit oscillatory behavior when approaching known performance boundaries. To mitigate the gap, we develop CausalEvolve, equipped with a causal scratchpad that leverages LLMs to identify and reason about guiding factors for evolution. At the beginning, CausalEvolve first identifies outcome-level factors that offer complementary inspirations in improving the target objective. During the evolution, CausalEvolve also inspects surprise patterns during the evolution and abductive reasoning to hypothesize new factors, which in turn offer novel directions. Through comprehensive experiments, we show that CausalEvolve effectively improves the evolutionary efficiency and discovers better solutions in 4 challenging open-ended scientific tasks.