Increased Compute Efficiency and the Diffusion of AI Capabilities

TL;DR

This paper discusses how improvements in hardware and algorithms reduce AI training costs, enabling wider access and faster development of advanced capabilities, while emphasizing the importance of responsible sharing and regulation.

Contribution

It introduces the concept of increasing compute efficiency and analyzes its effects on AI capability diffusion and competitive advantage.

Findings

Compute efficiency reduces training costs over time.

Wider access to AI capabilities increases as costs decrease.

Large investors maintain advantages by pioneering new capabilities.

Abstract

Training advanced AI models requires large investments in computational resources, or compute. Yet, as hardware innovation reduces the price of compute and algorithmic advances make its use more efficient, the cost of training an AI model to a given performance falls over time - a concept we describe as increasing compute efficiency. We find that while an access effect increases the number of actors who can train models to a given performance over time, a performance effect simultaneously increases the performance available to each actor. This potentially enables large compute investors to pioneer new capabilities, maintaining a performance advantage even as capabilities diffuse. Since large compute investors tend to develop new capabilities first, it will be particularly important that they share information about their AI models, evaluate them for emerging risks, and, more generally, make responsible development and release decisions. Further, as compute efficiency increases, governments will need to prepare for a world where dangerous AI capabilities are widely available - for instance, by developing defenses against harmful AI models or by actively intervening in the diffusion of particularly dangerous capabilities.