Small models, big threats: Characterizing safety challenges from low-compute AI models

TL;DR

This paper highlights the increasing risks posed by low-resource AI models, which have become more capable and dangerous, enabling harmful societal campaigns on consumer hardware, thus requiring new safety measures.

Contribution

It provides a comprehensive analysis of the performance trends of low-compute models and demonstrates their potential for societal harm, emphasizing the need for updated safety policies.

Findings

Model size needed for benchmarks has decreased over 10X in a year.

Most harmful campaigns can be run on consumer-grade hardware.

Current protections for high-compute models are insufficient for low-resource models.

Abstract

Artificial intelligence (AI) systems are revolutionizing fields such as medicine, drug discovery, and materials science; however, many technologists and policymakers are also concerned about the technology's risks. To date, most concrete policies around AI governance have focused on managing AI risk by considering the amount of compute required to operate or build a given AI system. However, low-compute AI systems are becoming increasingly more performant - and more dangerous. Driven by agentic workflows, parameter quantization, and other model compression techniques, capabilities once only achievable on frontier-level systems have diffused into low-resource models deployable on consumer devices. In this report, we profile this trend by downloading historical benchmark performance data for over 5,000 large language models (LLMs) hosted on HuggingFace, noting the model size needed to achieve competitive LLM benchmarks has decreased by more than 10X over the past year. We then simulate the computational resources needed for an actor to launch a series of digital societal harm campaigns - such as disinformation botnets, sexual extortion schemes, voice-cloning fraud, and others - using low-compute open-source models and find nearly all studied campaigns can easily be executed on consumer-grade hardware. This position paper argues that protection measures for high-compute models leave serious security holes for their low-compute counterparts, meaning it is urgent both policymakers and technologists make greater efforts to understand and address this emerging class of threats.