Understanding AI Data Repositories with Automatic Query Generation

TL;DR

This paper introduces techniques for automatic query generation from ingested data to evaluate and improve AI knowledge without human input, enhancing scalability and domain coverage.

Contribution

It presents novel methods for generating domain-agnostic queries to assess and extend AI knowledge, reducing reliance on human experts.

Findings

Queries can identify gaps and errors in AI knowledge.

Techniques enable scalable AI deployment across domains.

Future work will assess the effectiveness of these methods.

Abstract

We describe a set of techniques to generate queries automatically based on one or more ingested, input corpuses. These queries require no a priori domain knowledge, and hence no human domain experts. Thus, these auto-generated queries help address the epistemological question of how we know what we know, or more precisely in this case, how an AI system with ingested data knows what it knows. These auto-generated queries can also be used to identify and remedy problem areas in ingested material -- areas for which the knowledge of the AI system is incomplete or even erroneous. Similarly, the proposed techniques facilitate tests of AI capability -- both in terms of coverage and accuracy. By removing humans from the main learning loop, our approach also allows more effective scaling of AI and cognitive capabilities to provide (1) broader coverage in a single domain such as health or geology; and (2) more rapid deployment to new domains. The proposed techniques also allow ingested knowledge to be extended naturally. Our investigations are early, and this paper provides a description of the techniques. Assessment of their efficacy is our next step for future work.