TableQAKit: A Comprehensive and Practical Toolkit for Table-based Question Answering

TL;DR

TableQAKit is a comprehensive, open-source toolkit that unifies datasets, methods, and large language models for table-based question answering, enabling easier development and evaluation of TableQA systems.

Contribution

It introduces the first unified platform for TableQA that integrates datasets, methods, and LLMs, and provides a benchmark for evaluating LLMs in this domain.

Findings

Achieved new state-of-the-art results on some datasets.

Provides an interactive, user-friendly interface for TableQA research.

Includes an open-source, comprehensive benchmark for LLMs in TableQA.

Abstract

Table-based question answering (TableQA) is an important task in natural language processing, which requires comprehending tables and employing various reasoning ways to answer the questions. This paper introduces TableQAKit, the first comprehensive toolkit designed specifically for TableQA. The toolkit designs a unified platform that includes plentiful TableQA datasets and integrates popular methods of this task as well as large language models (LLMs). Users can add their datasets and methods according to the friendly interface. Also, pleasantly surprised using the modules in this toolkit achieves new SOTA on some datasets. Finally, \tableqakit{} also provides an LLM-based TableQA Benchmark for evaluating the role of LLMs in TableQA. TableQAKit is open-source with an interactive interface that includes visual operations, and comprehensive data for ease of use.