Toward Reusability of AI Models Using Dynamic Updates of AI Documentation

TL;DR

This paper proposes a methodology to improve AI model reusability by enabling dynamic updates of AI documentation, leveraging community data, and aligning with best practices to reduce lag and enhance documentation quality.

Contribution

It introduces an agile, data-driven approach for updating AI model documentation using community repositories and standard templates to boost reusability.

Findings

Correlation between AI model downloads and documentation quality.

Infrastructure for comparing documentation templates with community standards.

Quantification of documentation alignment with best practices.

Abstract

This work addresses the challenge of disseminating reusable artificial intelligence (AI) models accompanied by AI documentation (a.k.a., AI model cards). The work is motivated by the large number of trained AI models that are not reusable due to the lack of (a) AI documentation and (b) the temporal lag between rapidly changing requirements on AI model reusability and those specified in various AI model cards. Our objectives are to shorten the lag time in updating AI model card templates and align AI documentation more closely with current AI best practices. Our approach introduces a methodology for delivering agile, data-driven, and community-based AI model cards. We use the Hugging Face (HF) repository of AI models, populated by a subset of the AI research and development community, and the AI consortium-based Zero Draft (ZD) templates for the AI documentation of AI datasets and AI models, as our test datasets. We also address questions about the value of AI documentation for AI reusability. Our work quantifies the correlations between AI model downloads/likes (i.e., AI model reuse metrics) from the HF repository and their documentation alignment with the ZD documentation templates using tables of contents and word statistics (i.e., AI documentation quality metrics). Furthermore, our work develops the infrastructure to regularly compare AI documentation templates against community-standard practices derived from millions of uploaded AI models in the Hugging Face repository. The impact of our work lies in introducing a methodology for delivering agile, data-driven, and community-based standards for documenting AI models and improving AI model reuse.