The eDiscovery Medicine Show

TL;DR

This paper critiques the promotion of unvalidated eDiscovery tools and emphasizes the need for rigorous testing using established information retrieval methods to ensure effectiveness and scientific integrity.

Contribution

It highlights the parallels between historical medical practices and current eDiscovery technology promotion, advocating for scientific validation of tools.

Findings

Current eDiscovery tools often lack rigorous validation.

Promotion of unvetted 'AI' tools resembles pseudo-scientific medicine.

Call for adopting scientific testing protocols in eDiscovery.

Abstract

The practice of bloodletting gradually fell into disfavor as a growing body of scientific evidence showed its ineffectiveness and demonstrated the effectiveness of various pharmaceuticals for the prevention and treatment of certain diseases. At the same time, the patent medicine industry promoted ineffective remedies at medicine shows featuring entertainment, testimonials, and pseudo-scientific claims with all the trappings--but none of the methodology--of science. Today, many producing parties and eDiscovery vendors similarly promote obsolete technology as well as unvetted tools labeled "artificial intelligence" or "technology-assisted review," along with unsound validation protocols. This situation will end only when eDiscovery technologies and tools are subject to testing using the methods of information retrieval.