Leveraging Imperfection with MEDLEY A Multi-Model Approach Harnessing Bias in Medical AI

TL;DR

MEDLEY is a multi-model framework that leverages and documents biases and disagreements among AI models to enhance transparency and reasoning in medical diagnostics, challenging the traditional view of bias as purely negative.

Contribution

This work introduces MEDLEY, a novel multi-model approach that preserves model diversity and biases, providing a new perspective on AI imperfection in medical diagnostics.

Findings

Demonstrated preservation of diverse model outputs in synthetic cases

Showed how biases can be documented and potentially used as diagnostic resources

Created a proof-of-concept with over 30 large language models

Abstract

Bias in medical artificial intelligence is conventionally viewed as a defect requiring elimination. However, human reasoning inherently incorporates biases shaped by education, culture, and experience, suggesting their presence may be inevitable and potentially valuable. We propose MEDLEY (Medical Ensemble Diagnostic system with Leveraged diversitY), a conceptual framework that orchestrates multiple AI models while preserving their diverse outputs rather than collapsing them into a consensus. Unlike traditional approaches that suppress disagreement, MEDLEY documents model-specific biases as potential strengths and treats hallucinations as provisional hypotheses for clinician verification. A proof-of-concept demonstrator was developed using over 30 large language models, creating a minimum viable product that preserved both consensus and minority views in synthetic cases, making diagnostic uncertainty and latent biases transparent for clinical oversight. While not yet a validated clinical tool, the demonstration illustrates how structured diversity can enhance medical reasoning under clinician supervision. By reframing AI imperfection as a resource, MEDLEY offers a paradigm shift that opens new regulatory, ethical, and innovation pathways for developing trustworthy medical AI systems.