Towards Regulatable AI Systems: Technical Gaps and Policy Opportunities

TL;DR

This paper examines the technical challenges and opportunities in regulating AI systems, focusing on how AI experts can verify compliance with regulations and identifying areas for technical and interdisciplinary development.

Contribution

It analyzes existing public sector checklists to identify current capabilities and future needs for AI regulation verification methods.

Findings

Current checklists highlight technical gaps in AI compliance verification.

Technical innovations can enhance AI vetting processes.

Interdisciplinary approaches are necessary for comprehensive regulation.

Abstract

There is increasing attention being given to how to regulate AI systems. As governing bodies grapple with what values to encapsulate into regulation, we consider the technical half of the question: To what extent can AI experts vet an AI system for adherence to regulatory requirements? We investigate this question through the lens of two public sector procurement checklists, identifying what we can do now, what should be possible with technical innovation, and what requirements need a more interdisciplinary approach.