AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery

TL;DR

AutoSOTA is an automated system that accelerates AI research by reproducing, optimizing, and discovering new state-of-the-art models through a multi-agent architecture, reducing manual effort and fostering innovation.

Contribution

The paper introduces AutoSOTA, a novel end-to-end automated research system that advances the latest AI models to new SOTA levels with minimal human intervention.

Findings

AutoSOTA successfully discovered 105 new SOTA models surpassing original methods.

The system achieves an average of five hours per paper for replication and optimization.

Case studies demonstrate AutoSOTA's ability to identify architectural and algorithmic innovations.

Abstract

Artificial intelligence research increasingly depends on prolonged cycles of reproduction, debugging, and iterative refinement to achieve State-Of-The-Art (SOTA) performance, creating a growing need for systems that can accelerate the full pipeline of empirical model optimization. In this work, we introduce AutoSOTA, an end-to-end automated research system that advances the latest SOTA models published in top-tier AI papers to reproducible and empirically improved new SOTA models. We formulate this problem through three tightly coupled stages: resource preparation and goal setting; experiment evaluation; and reflection and ideation. To tackle this problem, AutoSOTA adopts a multi-agent architecture with eight specialized agents that collaboratively ground papers to code and dependencies, initialize and repair execution environments, track long-horizon experiments, generate and schedule optimization ideas, and supervise validity to avoid spurious gains. We evaluate AutoSOTA on recent research papers collected from eight top-tier AI conferences under filters for code availability and execution cost. Across these papers, AutoSOTA achieves strong end-to-end performance in both automated replication and subsequent optimization. Specifically, it successfully discovers 105 new SOTA models that surpass the original reported methods, averaging approximately five hours per paper. Case studies spanning LLM, NLP, computer vision, time series, and optimization further show that the system can move beyond routine hyperparameter tuning to identify architectural innovation, algorithmic redesigns, and workflow-level improvements. These results suggest that end-to-end research automation can serve not only as a performance optimizer, but also as a new form of research infrastructure that reduces repetitive experimental burden and helps redirect human attention toward higher-level scientific creativity.