SYNFIX: Automatically Fixing Syntax Errors using Compiler Diagnostics

TL;DR

SYNFIX is a machine-learning tool that significantly improves automatic syntax error correction in programming languages by leveraging large neural models and compiler diagnostics, aiding students and instructors.

Contribution

It introduces a novel neural architecture that uses multi-label classification and unsupervised pre-training to enhance syntax error fixing beyond existing methods.

Findings

Outperforms current state-of-the-art in fixing syntax errors.

Effective on longer and more complex programs.

Open-source implementation integrated with Visual Studio Code.

Abstract

Beginning programmers struggle with the complex grammar of modern programming languages like Java, and make lot of syntax errors. The diagnostic syntax error messages from compilers and IDEs are sometimes useful, but often the messages are cryptic and puzzling. Students could be helped, and instructors' time saved, by automated repair suggestions when dealing with syntax errors. Large samples of student errors and fixes are now available, offering the possibility of data-driven machine-learning approaches to help students fix syntax errors. Current machine-learning approaches do a reasonable job fixing syntax errors in shorter programs, but don't work as well even for moderately longer programs. We introduce SYNFIX, a machine-learning based tool that substantially improves on the state-of-the-art, by learning to use compiler diagnostics, employing a very large neural model that leverages unsupervised pre-training, and relying on multi-label classification rather than autoregressive synthesis to generate the (repaired) output. We describe SYNFIX's architecture in detail, and provide a detailed evaluation. We have built SYNFIX into a free, open-source version of Visual Studio Code; we make all our source code and models freely available.