Human-in-the-loop online multi-agent approach to increase trustworthiness in ML models through trust scores and data augmentation

TL;DR

This paper introduces a multi-agent system combining machine and human agents to enhance trustworthiness in ML models through trust scores and data augmentation, especially for safety-critical applications.

Contribution

It proposes a novel multi-agent framework that integrates trust scoring, anomaly filtering, human labeling, and data augmentation to improve model trustworthiness.

Findings

Improved accuracy on corrupted MNIST and FashionMNIST datasets.

Enhanced trust scores with minimal additional labels.

Effective data augmentation using geometric transformations.

Abstract

Increasing a ML model accuracy is not enough, we must also increase its trustworthiness. This is an important step for building resilient AI systems for safety-critical applications such as automotive, finance, and healthcare. For that purpose, we propose a multi-agent system that combines both machine and human agents. In this system, a checker agent calculates a trust score of each instance (which penalizes overconfidence and overcautiousness in predictions) using an agreement-based method and ranks it; then an improver agent filters the anomalous instances based on a human rule-based procedure (which is considered safe), gets the human labels, applies geometric data augmentation, and retrains with the augmented data using transfer learning. We evaluate the system on corrupted versions of the MNIST and FashionMNIST datasets. We get an improvement in accuracy and trust score with just few additional labels compared to a baseline approach.