Searching for non-English literature may be unnecessary for HTA Reports

TL;DR

This study finds that non-English studies rarely change the conclusions of health technology assessments, suggesting that focusing on English-language research is usually sufficient.

Contribution

The study provides evidence that language restrictions in health technology assessments may be justified due to minimal impact from non-English studies.

Findings

54% of HTA reports included non-English publications, but only 6% of these changed conclusions when excluded.

Non-English studies were often the only available data, leading to a lack of analysable information after exclusion.

In most cases, non-English studies had little influence on the conclusions of HTA reports.

Abstract

Health technology assessment (HTA) reports are based on comprehensive information retrieval. Current standards discourage the use of search restrictions, such as publication date and language. Given limited resources, it was unclear whether the effort invested in screening and translating studies published in languages other than English provided relevant additional information compared with the inclusion of English-language publications alone. We therefore analysed the impact of non-English publications on the conclusions of HTA reports produced by the German HTA agency, the Institute for Quality and Efficiency in Health Care (IQWiG).

We determined whether non-English publications were included in all HTA reports on non-drug interventions and on selected drug interventions (search period: 01/2011 to 08/2018). If at least one non-English publication was included, we assessed for each endpoint whether or not the exclusion of non-English publications changed the conclusion. If a non-English publication did not contain information relevant to the HTA report, we classified the publication as “not relevant”.

Of 70 HTA reports, 38 (54%) included 128 non-English publications. In 4 reports (6%) with 50 endpoints investigated in 39 PICO questions, the exclusion of a total of 10 non-English publications led to a change in the conclusions for 13 endpoints (8 PICO questions). This was largely due to the fact that in many cases, non-English publications were the predominant or only literature available, resulting in a lack of analysable data after their exclusion.

In general, studies published in non-English languages have little influence on the conclusions of HTA reports. For the vast majority of topics, a language restriction to English seems justified. Studies published in non-English languages may be useful in exceptional cases, for example when an intervention is only available in certain countries.

Figures