Computer Environments Elicit General Agentic Intelligence in LLMs

TL;DR

This paper introduces LLM-in-Sandbox, a minimal computer environment that enhances large language models' general capabilities and efficiency without additional training, highlighting the environment's role in eliciting agentic intelligence.

Contribution

The study systematically investigates how simple computer environments can elicit general capabilities in LLMs and develops training methods to harness these interactions.

Findings

LLM-in-Sandbox improves performance across multiple domains by up to 15.5%.

Models reduce token consumption by up to 8 times in the sandbox environment.

Training with LLM-in-Sandbox-RL enables weaker models to utilize environmental interactions.

Abstract

Agentic intelligence in large language models (LLMs) requires not only model intrinsic capabilities but also interactions with external environments. Equipping LLMs with computers now represents a prevailing trend. However, the computer environment's intrinsic value has not been systematically investigated, particularly its potential to elicit general capabilities. Here we introduce LLM-in-Sandbox, which virtualizes the computer as a code sandbox with only basic functionalities, and demonstrate that this minimal setting elicits computer-based meta-capabilities for general task solving: external resource access, file management, and code execution. Without additional training, strong models achieve substantial gains (up to 15.5%) across mathematics, physics, chemistry, biomedicine, long-context understanding, and instruction following, while reducing token consumption by up to 8 times. Furthermore, we develop LLM-in-Sandbox-RL to train models exclusively on non-agentic data within the sandbox, empowering weaker models to harness the environment and internalize these interactions. Our results demonstrate that computer environments elicit general intelligence, yield efficiency gains, and can be harnessed through training, serving as a promising foundation for generalist agents.