A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization
Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Minjoon Seo

TL;DR
This paper introduces SQLova, a novel NL2SQL model leveraging BERT with table-aware contextualization, achieving near-human performance on WikiSQL and outperforming previous methods significantly.
Contribution
It presents SQLova, the first model to combine BERT with table-aware contextualization for NL2SQL, surpassing prior state-of-the-art results and providing comprehensive analysis of dataset and model performance.
Findings
SQLova achieves 8.2% and 2.5% improvements in logical form and execution accuracy.
The model exceeds human performance by 1.3% in execution accuracy.
BERT with seq2seq decoder performs poorly, highlighting the importance of model design.
Abstract
We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to achieve human performance in WikiSQL dataset. We revisit and discuss diverse popular methods in NL2SQL literature, take a full advantage of BERT {Devlin et al., 2018) through an effective table contextualization method, and coherently combine them, outperforming the previous state of the art by 8.2% and 2.5% in logical form and execution accuracy, respectively. We particularly note that BERT with a seq2seq decoder leads to a poor performance in the task, indicating the importance of a careful design when using such large pretrained models. We also provide a comprehensive analysis on the dataset and our model, which can be helpful for designing future NL2SQL datsets and models. We especially show that our model's performance is near the upper bound in WikiSQL, where we observe that a large portion of the evaluation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Software Engineering Research
MethodsLinear Layer · Sigmoid Activation · Tanh Activation · Residual Connection · Attention Dropout · Linear Warmup With Linear Decay · Weight Decay · Refunds@Expedia|||How do I get a full refund from Expedia? · Dense Connections · Adam
