Safety versus performance: How multi-objective learning reduces barriers to market entry

TL;DR

This paper explores how multi-objective learning can reduce barriers for new companies entering the market for large language models by considering reputational risks.

Contribution

The paper introduces a multi-objective regression framework to analyze market entry and shows how reputational damage reduces data requirements for new entrants.

Findings

New companies need significantly fewer data points to enter the market compared to the incumbent's dataset size.

Reputational damage disproportionately affects incumbents, reducing entry barriers for new firms.

Scaling laws in multi-objective environments show slower performance gains with larger datasets.

Abstract

The development of large language models has given rise to emerging markets where companies offer models as a service and compete for user usage. A concern is that the accumulation of data and compute by incumbents creates insurmountable barriers to entry for new companies. We develop a multi-objective high-dimensional regression framework to study market entry, focusing on a phenomenon which challenges this intuition. Our framework captures the reputational damage that companies face due to models’ safety violations. We show how the incumbents face greater threat of reputational damage than new companies, which reduces the amount of data the new company needs to enter the market. We quantify this reduction as a function of the incumbent’s data size.

Emerging marketplaces for large language models and other large-scale machine learning models appear to exhibit market concentration, which has raised concerns about whether there are insurmountable barriers to entry in such markets. In this work, we study this issue from both an economic and an algorithmic point of view, focusing on a phenomenon that reduces barriers to entry. Specifically, an incumbent company risks reputational damage unless its model is sufficiently aligned with safety objectives, whereas a new company can more easily avoid reputational damage. To study this issue formally, we define a multi-objective high-dimensional regression framework that captures reputational damage, and we characterize the number of data points that a new company needs to enter the market. Our results demonstrate how multi-objective considerations can fundamentally reduce barriers to entry—the required number of data points can be significantly smaller than the incumbent company’s dataset size. En route to proving these results, we develop scaling laws for high-dimensional linear regression in multi-objective environments, showing that the scaling rate becomes slower when the dataset size is large, which could be of independent interest.

Figures