Interactive Evidence Maps for Visualizing and Understanding Systematic Reviews

TL;DR

This paper introduces interactive evidence maps that utilize large language models to structure and visualize systematic review data, enhancing exploration, transparency, and gap detection.

Contribution

The paper presents a novel interactive visualization tool that leverages large language models to create explorable knowledge maps from systematic review data.

Findings

Evidence maps improve transparency and exploration of review data.

They reveal patterns and gaps not easily seen in traditional summaries.

The approach is demonstrated with a review of pedagogical agents in education.

Abstract

Systematic reviews provide comprehensive syntheses of research fields. As a result, systematic reviews often emphasize synthesizing across the large bodies of literature rather than just describing the studies from which the conclusions were drawn. This risks an incomplete description of the sample - encouraging overgeneralization of the findings, obscuring connections between existing work, or overshadowing gaps in the literature. To address this challenge, we introduce interactive evidence maps; an accessible visualization tool that enables researchers to explore, filter, and analyze review data dynamically. Our approach leverages large language models to extract topic models that structure heterogeneous review data into an interactive, explorable knowledge map that supports deeper inspection beyond static tables and figures. We demonstrate the usefulness of interactive evidence maps using data from a published scoping review of pedagogical agents in K-12 education, and compare the results of the evidence map to those reported in the scoping review. Results show that interactive evidence maps complement traditional syntheses by enhancing transparency, supporting exploratory analysis, and revealing patterns and gaps that may not be easy to detect through narrative summaries alone.