Inter- and intra-observer agreement in ultrasound diagnosis of steatotic liver disease: implications for screening in resource-limited settings

TL;DR

This study shows that ultrasound diagnosis of liver disease has inconsistent results between and within observers, suggesting the need for better training and quality control in high-risk populations.

Contribution

The study quantifies inter- and intra-observer variability in ultrasound-based SLD diagnosis in a high-risk cohort.

Findings

Inter-observer agreement between a radiologist and technicians was slight to fair.

Intra-observer agreement was moderate to substantial for some observers.

The results highlight the need for quality control in ultrasound-based SLD screening.

Abstract

Steatotic liver disease (SLD), which is associated with increased risk of cancer-related mortality, needs timely and cost-effective detection. Although liver biopsy remains the diagnostic gold standard, its invasiveness and high-cost limit widespread use. Ultrasound is a practical and affordable alternative. We evaluated inter- and intra-observer agreement for ultrasound-based diagnosis of SLD using images from the Chile Biliary Longitudinal Study (Chile BiLS), a cohort of women with gallstones. These women have a high burden of obesity and related metabolic disorders, putting them at higher risk for SLD. A radiologist (observer 1) reviewed a randomly selected subset of 425 baseline images and compared them with the original readings from Chile BiLS radiology technicians. To assess intra-observer reproducibility, observer 1 reanalyzed 34 blinded duplicates, and two Chile BiLS radiology technicians (observers 2 and 3) independently reviewed these images. Observer 2 then re-reviewed the 34 images to assess intra-observer agreement. Agreement was analyzed using kappa and percent agreement. Observer 1 had slight inter-observer agreement (kappa: 0.12; 95% CI 0.08–0.15, p < 0.001; percent agreement: 41.0%), while observers 2 and 3 showed fair agreement (kappa: 0.29: 95% CI 0.11–0.58, p < 0.05; percent agreement: 64.7% and kappa: 0.32: 95% CI 0.06–0.58, p < 0.05; percent agreement: 63.6%, respectively). Intra-observer agreement was moderate for observer 1 (kappa: 0.45; 95% CI 0.08–0.82, p < 0.05; percent agreement: 81.3%), and substantial for observer 2 (kappa: 0.64; 95% CI 0.37–0.90, p < 0.001; percent agreement: 81.8%). Our findings highlight variability in ultrasound interpretation, underscoring the necessity of inter- and intra-observer comparisons for optimal diagnosis and quality control to enhance diagnostic consistency in high-risk populations.

Figures