CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
Zi-Han Wang, Lam Nguyen, Zhengyang Zhao, Mengyue Yang, Chengwei Qin, Yujiu Yang, Linyi Yang

TL;DR
CreativeBench provides a rigorous, quantitative benchmark for evaluating machine creativity in code generation, revealing insights into model behaviors and introducing EvoRePE to enhance creativity.
Contribution
The paper introduces CreativeBench, a novel benchmark for assessing machine creativity, and proposes EvoRePE, a strategy to improve creative outputs during inference.
Findings
Scaling improves combinatorial creativity but not exploration.
Larger models tend to become more correct but less divergent.
Reasoning skills mainly aid constrained exploration.
Abstract
The saturation of high-quality pre-training data has shifted research focus toward evolutionary systems capable of continuously generating novel artifacts, leading to the success of AlphaEvolve. However, the progress of such systems is hindered by the lack of rigorous, quantitative evaluation. To tackle this challenge, we introduce CreativeBench, a benchmark for evaluating machine creativity in code generation, grounded in a classical cognitive framework. Comprising two subsets -- CreativeBench-Combo and CreativeBench-Explore -- the benchmark targets combinatorial and exploratory creativity through an automated pipeline utilizing reverse engineering and self-play. By leveraging executable code, CreativeBench objectively distinguishes creativity from hallucination via a unified metric defined as the product of quality and novelty. Our analysis of state-of-the-art models reveals distinct…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Games · Music Technology and Sound Studies · Machine Learning in Materials Science
