The doctor will polygraph you now: ethical concerns with AI for fact-checking patients

TL;DR

This paper explores ethical issues related to AI-based fact-checking in healthcare, highlighting risks of bias, privacy concerns, and impacts on trust, and proposes mitigation strategies.

Contribution

It identifies key ethical challenges in clinical AI for social behavior verification and offers recommendations to mitigate potential harms.

Findings

Simulated misuse of verification system reveals bias against patient-reported data.

AI bias favors multi-dimensional data over patient voice samples.

Recommendations provided for ethical AI deployment in healthcare.

Abstract

Artificial intelligence (AI) methods have been proposed for the prediction of social behaviors which could be reasonably understood from patient-reported information. This raises novel ethical concerns about respect, privacy, and control over patient data. Ethical concerns surrounding clinical AI systems for social behavior verification can be divided into two main categories: (1) the potential for inaccuracies/biases within such systems, and (2) the impact on trust in patient-provider relationships with the introduction of automated AI systems for fact-checking, particularly in cases where the data/models may contradict the patient. Additionally, this report simulated the misuse of a verification system using patient voice samples and identified a potential LLM bias against patient-reported information in favor of multi-dimensional data and the outputs of other AI methods (i.e., AI self-trust). Finally, recommendations were presented for mitigating the risk that AI verification methods will cause harm to patients or undermine the purpose of the healthcare system.