Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies

TL;DR

This paper evaluates the Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis, especially when few studies are involved, showing the modified method provides more reliable inference in such cases.

Contribution

The study compares standard and modified HKSJ methods through simulations, recommending the modified approach for meta-analyses with few studies and varying study precisions.

Findings

Modified mKH yields more conservative and accurate error rates.

Standard HKSJ performs well with similar study precisions.

Differences are most notable with few, heterogenous studies.

Abstract

BACKGROUND: Random-effects meta-analysis is commonly performed by first deriving an estimate of the between-study variation, the heterogeneity, and subsequently using this as the basis for combining results, i.e., for estimating the effect, the figure of primary interest. The heterogeneity variance estimate however is commonly associated with substantial uncertainty, especially in contexts where there are only few studies available, such as in small populations and rare diseases. METHODS: Confidence intervals and tests for the effect may be constructed via a simple normal approximation, or via a Student-t distribution, using the Hartung-Knapp-Sidik-Jonkman (HKSJ) approach, which additionally uses a refined estimator of variance of the effect estimator. The modified Knapp-Hartung method (mKH) applies an ad hoc correction and has been proposed to prevent counterintuitive effects and to yield more conservative inference. We performed a simulation study to investigate the behaviour of the standard HKSJ and modified mKH procedures in a range of circumstances, with a focus on the common case of meta-analysis based on only a few studies. RESULTS: The standard HKSJ procedure works well when the treatment effect estimates to be combined are of comparable precision, but nominal error levels are exceeded when standard errors vary considerably between studies (e.g. due to variations in study size). Application of the modification on the other hand yields more conservative results with error rates closer to the nominal level. Differences are most pronounced in the common case of few studies of varying size or precision. CONCLUSIONS: Use of the modified mKH procedure is recommended, especially when only a few studies contribute to the meta-analysis and the involved studies' precisions (standard errors) vary.