Natural Language Models for Data Visualization Utilizing nvBench Dataset

TL;DR

This paper explores transformer-based natural language models, including BERT and T5, for translating natural language queries into data visualization commands using the nvBench dataset.

Contribution

It introduces a sequence-to-sequence transformer approach for translating natural language into visualization commands, comparing different large language models.

Findings

Transformer models can effectively translate natural language into visualization commands.

BERT-based encoders show promising performance in this task.

T5 models provide a competitive alternative for natural language to visualization translation.

Abstract

Translation of natural language into syntactically correct commands for data visualization is an important application of natural language models and could be leveraged to many different tasks. A closely related effort is the task of translating natural languages into SQL queries, which in turn could be translated into visualization with additional information from the natural language query supplied\cite{Zhong:2017qr}. Contributing to the progress in this area of research, we built natural language translation models to construct simplified versions of data and visualization queries in a language called Vega Zero. In this paper, we explore the design and performance of these sequence to sequence transformer based machine learning model architectures using large language models such as BERT as encoders to predict visualization commands from natural language queries, as well as apply available T5 sequence to sequence models to the problem for comparison.