A systematic audit of transparency and validation disclosure in commercial veterinary artificial intelligence

TL;DR

This study audits transparency in commercial veterinary AI tools, finding significant gaps in public disclosure of validation and data.

Contribution

The paper introduces a systematic audit framework to assess transparency in veterinary AI, revealing a critical 'Transparency Gap' in the industry.

Findings

The mean unweighted transparency score across 71 AI products was 6.4%, with most vendors failing to disclose any metrics.

Diagnostic Imaging tools scored higher in transparency (13.1%) compared to Generative and Ambient tools (1.8%).

Only 2.1% of generative AI vendors provided validation evidence, and just 1.4% disclosed training data demographics.

Abstract

To systematically identify the commercial market for clinical artificial intelligence (AI) products in veterinary medicine and audit their public documentation for transparency using a standardized, evidence-based instrument.

A cross-sectional systematic audit of commercial AI tools was completed via a multi-channel search. Inclusion criteria required commercially available products with explicit AI claims and clinical functionality; administrative and direct-to-consumer tools were excluded. Publicly available documentation was archived and evaluated using a 25-point framework adapted from FDA and GMLP guidelines to assess data provenance, validation, safety, and usability.

Seventy-one AI products, available in the North American market were included, comprising Generative and Ambient (n = 47), Diagnostic Imaging (n = 19), and Specialized tools (n = 5). The mean unweighted transparency score across the cohort was 6.4%. Notably, 63.3% (n = 45) of vendors failed to disclose a single metric. Diagnostic Imaging tools achieved a higher mean risk-weighted transparency score (13.1%) compared to Generative and Ambient tools (1.8%). While 36.8% of imaging vendors provided peer-reviewed or internal validation evidence, only 2.1% of generative vendors did so. Only one vendor (1.4%) disclosed training data signalment (species, breed, age, sex) or subgroup performance.

The commercial veterinary AI market operates with systemic opacity. This audit reveals a significant “Transparency Gap”—a divergence where the sophisticated clinical capabilities marketed to veterinarians far exceed the publicly available evidence required to validate them. A significant gap exists between maturing imaging applications and unvalidated generative tools. The universal failure to report training demographics renders independent assessment of algorithmic bias impossible.

Veterinarians currently bear the legal and ethical burden of validating AI tools without access to necessary performance data. The implementation of standardized transparency frameworks is urgently required to support evidence-based product selection and prevent patient harm from unvalidated technologies.

Figures