The Achilles Heel of AI: Fundamentals of Risk-Aware Training Data for High-Consequence Models

TL;DR

This paper presents smart-sizing, a risk-aware training data strategy that improves high-consequence AI models by focusing on label diversity and model-guided selection, reducing data needs while enhancing rare event detection.

Contribution

It introduces Adaptive Label Optimization (ALO), combining triage, disagreement analysis, and feedback to prioritize impactful labels, advancing data efficiency in critical AI applications.

Findings

Models trained on 20-40% of curated data match or outperform full datasets.

Enhanced rare-class recall and edge-case generalization.

Embedded audit tools reveal latent labeling errors affecting evaluation.

Abstract

AI systems in high-consequence domains such as defense, intelligence, and disaster response must detect rare, high-impact events while operating under tight resource constraints. Traditional annotation strategies that prioritize label volume over informational value introduce redundancy and noise, limiting model generalization. This paper introduces smart-sizing, a training data strategy that emphasizes label diversity, model-guided selection, and marginal utility-based stopping. We implement this through Adaptive Label Optimization (ALO), combining pre-labeling triage, annotator disagreement analysis, and iterative feedback to prioritize labels that meaningfully improve model performance. Experiments show that models trained on 20 to 40 percent of curated data can match or exceed full-data baselines, particularly in rare-class recall and edge-case generalization. We also demonstrate how latent labeling errors embedded in training and validation sets can distort evaluation, underscoring the need for embedded audit tools and performance-aware governance. Smart-sizing reframes annotation as a feedback-driven process aligned with mission outcomes, enabling more robust models with fewer labels and supporting efficient AI development pipelines for frontier models and operational systems.