To Type or Not to Type? A Systematic Comparison of the Software Quality of JavaScript and TypeScript Applications on GitHub

TL;DR

This study empirically compares JavaScript and TypeScript applications on GitHub across multiple quality metrics, revealing that TypeScript improves code quality and understandability but does not significantly reduce bug proneness or resolution time.

Contribution

It provides the first large-scale empirical analysis comparing JS and TS software quality across four facets, highlighting nuanced effects of type safety on bug metrics.

Findings

TypeScript apps have better code quality and understandability.

Bug proneness and fix time are not significantly lower in TypeScript.

Reduced use of `any` type correlates with improved quality metrics.

Abstract

JavaScript (JS) is one of the most popular programming languages, and widely used for web apps and even backend development. Due to its dynamic nature, however, JS applications often have a reputation for poor software quality. As a type-safe superset of JavaScript, TypeScript (TS) offers features to address this. However, there is currently insufficient empirical evidence to broadly support the claim that TS apps exhibit better software quality than JS apps. We therefore conducted a repository mining study based on 604 GitHub projects (299 for JS, 305 for TS) with over 16M LoC and collected four facets of software quality: a) code quality (# of code smells per LoC), b) code understandability (cognitive complexity per LoC), c) bug proneness (bug fix commit ratio), and d) bug resolution time (mean time a bug issue is open). For TS, we also collected how frequently the type-safety ignoring `any` type was used. The analysis indicates that TS apps exhibit significantly better code quality and understandability than JS apps. Contrary to expectations, however, bug proneness and bug resolution time of our TS sample were not significantly lower than for JS: mean bug fix commit ratio was more than 60% larger (0.126 vs. 0.206), and TS projects needed on average more than an additional day to fix bugs (31.86 vs. 33.04 days). Furthermore, reducing the usage of the `any` type in TS apps was significantly correlated with all metrics except bug proneness (Spearman's rho between 0.17 and 0.26). Our results indicate that the perceived positive influence of TypeScript for avoiding bugs in comparison to JavaScript may be more complicated than assumed. While using TS seems to have benefits, it does not automatically lead to less and easier to fix bugs. However, more research is needed in this area, especially concerning the potential influence of project complexity and developer experience.