TL;DR
This paper enhances table entailment understanding by adapting TAPAS with intermediate pre-training on a large automatically generated dataset, improving efficiency and setting new state-of-the-art results on TabFact and SQA datasets.
Contribution
It introduces a novel intermediate pre-training step with a large dataset for table entailment, improving model performance and efficiency.
Findings
Achieved state-of-the-art on TabFact and SQA datasets.
Data augmentation with automatically generated examples is effective.
Table pruning improves training and prediction efficiency.
Abstract
Table entailment, the binary classification task of finding if a sentence is supported or refuted by the content of a table, requires parsing language and table structure as well as numerical and discrete reasoning. While there is extensive work on textual entailment, table entailment is less well studied. We adapt TAPAS (Herzig et al., 2020), a table-based BERT model, to recognize entailment. Motivated by the benefits of data augmentation, we create a balanced dataset of millions of automatically created training examples which are learned in an intermediate step prior to fine-tuning. This new data is not only useful for table entailment, but also for SQA (Iyyer et al., 2017), a sequential table QA task. To be able to use long examples as input of BERT models, we evaluate table pruning techniques as a pre-processing step to drastically improve the training and prediction efficiency at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
- 🤗google/tapas-base-finetuned-sqamodel· 1.3k dl· ♡ 71.3k dl♡ 7
- 🤗google/tapas-base-finetuned-tabfactmodel· 389 dl· ♡ 1389 dl♡ 1
- 🤗google/tapas-base-finetuned-wikisql-supervisedmodel· 359 dl· ♡ 9359 dl♡ 9
- 🤗google/tapas-base-finetuned-wtqmodel· 8.8k dl· ♡ 2348.8k dl♡ 234
- 🤗google/tapas-basemodel· 4.8k dl· ♡ 104.8k dl♡ 10
- 🤗google/tapas-large-finetuned-sqamodel· 81k dl· ♡ 781k dl♡ 7
- 🤗google/tapas-large-finetuned-tabfactmodel· 467 dl· ♡ 4467 dl♡ 4
- 🤗google/tapas-large-finetuned-wikisql-supervisedmodel· 16 dl· ♡ 616 dl♡ 6
- 🤗google/tapas-large-finetuned-wtqmodel· 1.3k dl· ♡ 1481.3k dl♡ 148
- 🤗google/tapas-largemodel· 9 dl· ♡ 39 dl♡ 3
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsPruning · Linear Layer · TAPAS · Adam · Softmax · Refunds@Expedia|||How do I get a full refund from Expedia? · Dense Connections · Dropout · Linear Warmup With Linear Decay · Layer Normalization
