Remote Labor Index: Measuring AI Automation of Remote Work

TL;DR

The paper introduces the Remote Labor Index (RLI), a benchmark to evaluate AI's practical automation capabilities across sectors, revealing current AI agents perform poorly, with only 2.5% automation rate, thus informing discussions on AI's economic impact.

Contribution

It presents the RLI as a new multi-sector benchmark for assessing AI automation in real-world remote work scenarios, bridging research progress with economic implications.

Findings

AI agents perform near the floor on RLI

Highest-performing agent achieves 2.5% automation rate

RLI provides empirical basis for AI automation discussions

Abstract

AIs have made rapid progress on research-oriented benchmarks of knowledge and reasoning, but it remains unclear how these gains translate into economic value and automation. To measure this, we introduce the Remote Labor Index (RLI), a broadly multi-sector benchmark comprising real-world, economically valuable projects designed to evaluate end-to-end agent performance in practical settings. AI agents perform near the floor on RLI, with the highest-performing agent achieving an automation rate of 2.5%. These results help ground discussions of AI automation in empirical evidence, setting a common basis for tracking AI impacts and enabling stakeholders to proactively navigate AI-driven labor automation.