Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Study

TL;DR

This empirical study investigates the security and deployment characteristics of publicly accessible large language models, revealing widespread vulnerabilities, insecure configurations, and inconsistent access controls across various frameworks.

Contribution

It provides the first large-scale measurement and analysis of public-facing LLM deployments, highlighting systemic security risks and exposing the need for improved deployment practices.

Findings

Over 40% of API endpoints used plain HTTP.

More than 210,000 endpoints lacked valid TLS metadata.

Some frameworks responded to over 35% unauthenticated requests.

Abstract

Large language models (LLMs) are increasingly deployed through open-source and commercial frameworks, enabling individuals and organizations to self-host advanced LLM capabilities. As LLM deployments become prevalent, particularly in industry, ensuring their secure and reliable operation has become a critical issue. However, insecure defaults and misconfigurations often expose LLM services to the public internet, posing serious security and system engineering risks. This study conducted a large-scale empirical investigation of public-facing LLM deployments, focusing on the prevalence of services, exposure characteristics, systemic vulnerabilities, and associated risks. Through internet-wide measurements, we identified 320,102 public-facing LLM services across 15 frameworks and extracted 158 unique API endpoints, categorized into 12 functional groups based on functionality and security risk. Our analysis found that over 40% of endpoints used plain HTTP, and over 210,000 endpoints lacked valid TLS metadata. API exposure was highly inconsistent: some frameworks, such as Ollama, responded to over 35% of unauthenticated API requests, with about 15% leaking model or system information, while other frameworks implemented stricter controls. We observed widespread use of insecure protocols, poor TLS configurations, and unauthenticated access to critical operations. These security risks, such as model leakage, system compromise, and unauthorized access, are pervasive and highlight the need for a secure-by-default framework and stronger deployment practices.