Developing a Risk Identification Framework for Foundation Model Uses

TL;DR

This paper proposes a structured framework to help practitioners identify relevant risks associated with foundation model applications, addressing a gap in practical risk management guidance.

Contribution

It introduces a set of design requirements and an initial framework for risk identification tailored to foundation model uses, inspired by usage governance principles.

Findings

Framework addresses key risk identification challenges

Demonstration with a practical use case

Guidance for practitioners on risk relevance

Abstract

As foundation models grow in both popularity and capability, researchers have uncovered a variety of ways that the models can pose a risk to the model's owner, user, or others. Despite the efforts of measuring these risks via benchmarks and cataloging them in AI risk taxonomies, there is little guidance for practitioners on how to determine which risks are relevant for a given foundation model use. In this paper, we address this gap and develop requirements and an initial design for a risk identification framework. To do so, we look to prior literature to identify challenges for building a foundation model risk identification framework and adapt ideas from usage governance to synthesize four design requirements. We then demonstrate how a candidate framework can addresses these design requirements and provide a foundation model use example to show how the framework works in practice for a small subset of risks.