Expanding the scope of statistical computing: Training statisticians to be software engineers

TL;DR

This paper presents a graduate course designed to train statisticians in software engineering skills, emphasizing programming practices, software design, algorithms, and tools to meet the evolving demands of statistical software development.

Contribution

It introduces a novel curriculum that integrates software engineering principles into statistical education, preparing students for software development roles.

Findings

Students gain practical software engineering skills.

The course enhances understanding of algorithms and data structures.

Students complete a semester-long software project.

Abstract

Traditionally, statistical computing courses have taught the syntax of a particular programming language or specific statistical computation methods. Since the publication of Nolan and Temple Lang (2010), we have seen a greater emphasis on data wrangling, reproducible research, and visualization. This shift better prepares students for careers working with complex datasets and producing analyses for multiple audiences. But, we argue, statisticians are now often called upon to develop statistical software, not just analyses, such as R packages implementing new analysis methods or machine learning systems integrated into commercial products. This demands different skills. We describe a graduate course that we developed to meet this need by focusing on four themes: programming practices; software design; important algorithms and data structures; and essential tools and methods. Through code review and revision, and a semester-long software project, students practice all the skills of software engineering. The course allows students to expand their understanding of computing as applied to statistical problems while building expertise in the kind of software development that is increasingly the province of the working statistician. We see this as a model for the future evolution of the computing curriculum in statistics and data science.