Code and Structure Editing for Teaching: A Case Study in using Bibliometrics to Guide Computer Science Research

TL;DR

This paper investigates why structure editors are underused in education by conducting a bibliometric analysis of computer science literature, highlighting the need for empirical evidence and user-centered design in tool development.

Contribution

It provides a systematic review and bibliometric analysis of research on structure editors, proposing hypotheses about barriers to adoption and offering methodological guidance for future research.

Findings

Lack of empirical evidence hinders adoption in education.

Existing tools often lack user-centered design principles.

Bibliometric tools can aid in understanding research trends and tool coverage.

Abstract

Structure or projectional editors are a well-studied concept among researchers and some practitioners. They have the huge advantage of preventing syntax and in some cases type errors, and aid the discovery of syntax by users unfamiliar with a language. This begs the question: why are they not widely used in education? To answer this question we performed a systematic review of 57 papers and performed a bibliometric analysis which extended to 381 papers. From these we generated two hypotheses: (1) a lack of empirical evidence prevents educators from committing to this technology, and (2) existing tools have not been designed based on actual user needs as they would be if human-centered design principles were used. Given problems we encountered with existing resources to support a systematic review, and the role of bibliometric tools in overcoming those obstacles, we also detail our methods so that they may be used as a guide for researchers or graduate students unfamiliar with bibliometrics. In particular, we report on which tools provide reliable and plentiful information in the field of computer science, and which have insufficient coverage and interoperability issues.