Statistical Scenario Modelling and Lookalike Distributions for Multi-Variate AI Risk

TL;DR

This paper introduces statistically rigorous methods for modeling AI risk using scenario modeling and lookalike distributions, enabling holistic risk assessment and impact estimation despite limited data.

Contribution

It presents novel application of statistical techniques like Markov chains, copulas, and Monte Carlo simulations for AI risk modeling and introduces lookalike distributions to estimate AI impacts without direct data.

Findings

Effective holistic AI risk modeling demonstrated

Lookalike distributions provide impact estimates without direct data

Benchmarking of AI risk via scenario simulations successfully conducted

Abstract

Evaluating AI safety requires statistically rigorous methods and risk metrics for understanding how the use of AI affects aggregated risk. However, much AI safety literature focuses upon risks arising from AI models in isolation, lacking consideration of how modular use of AI affects risk distribution of workflow components or overall risk metrics. There is also a lack of statistical grounding enabling sensitisation of risk models in the presence of absence of AI to estimate causal contributions of AI. This is in part due to the dearth of AI impact data upon which to fit distributions. In this work, we address these gaps in two ways. First, we demonstrate how scenario modelling (grounded in established statistical techniques such as Markov chains, copulas and Monte Carlo simulation) can be used to model AI risk holistically. Second, we show how lookalike distributions from phenomena analogous to AI can be used to estimate AI impacts in the absence of directly observable data. We demonstrate the utility of our methods for benchmarking cumulative AI risk via risk analysis of a logistic scenario simulations.