Early Discoveries of Algorithmist I: Promise of Provable Algorithm Synthesis at Scale
Janardhan Kulkarni

TL;DR
This paper presents Algorithmist, an autonomous agent leveraging LLMs to synthesize provably sound algorithms tailored to specific datasets, demonstrating practical effectiveness and uncovering new insights in algorithm design and verification.
Contribution
It introduces a novel LLM-based research agent capable of on-the-fly provable algorithm synthesis, bridging theory and practice without prior distributional assumptions.
Findings
Produced provably sound algorithms for data privacy and clustering tasks
Uncovered a subtle proof bug in prior work
Generated research-quality algorithmic artifacts tailored to datasets
Abstract
Designing algorithms with provable guarantees that also work well in practice remains difficult, requiring both mathematical reasoning and careful implementation. Existing approaches that bridge worst-case theory and empirical performance, such as beyond-worst-case analysis and data-driven algorithm selection, typically assume prior distributional knowledge or restrict attention to a fixed pool of algorithms. Recent progress in LLMs suggests a new possibility: provable algorithm synthesis on the fly. To study this, we built Algorithmist, an autonomous researcher agent on top of GitHub Copilot that runs a multi-agent research-and-review loop, with separate stages for idea generation, algorithm and proof development, proof-guided implementation, and review of proofs, code, and their alignment. We evaluate Algorithmist on research-level tasks in private data analysis and clustering. When…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
