The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs

TL;DR

This paper introduces the AIQ Meta-Testbed, a pragmatic framework aimed at bridging the gap between academic AI testing methods and industrial quality assurance needs, emphasizing testing as a key aspect.

Contribution

It proposes a working definition of AI quality and presents ongoing efforts to develop a meta-testbed for AI testing applicable across domains.

Findings

Defined a pragmatic approach to AI quality assurance

Developing the AIQ Meta-Testbed for industrial testing needs

Bridging academic and industrial AI testing practices

Abstract

AI solutions seem to appear in any and all application domains. As AI becomes more pervasive, the importance of quality assurance increases. Unfortunately, there is no consensus on what artificial intelligence means and interpretations range from simple statistical analysis to sentient humanoid robots. On top of that, quality is a notoriously hard concept to pinpoint. What does this mean for AI quality? In this paper, we share our working definition and a pragmatic approach to address the corresponding quality assurance with a focus on testing. Finally, we present our ongoing work on establishing the AIQ Meta-Testbed.