Knowledge Base Question Answering for Space Debris Queries

TL;DR

This paper presents a system for answering complex natural language questions about space debris using a knowledge base, leveraging a pipeline approach and semi-synthetic data generation to improve performance with limited training data.

Contribution

The work introduces a novel pipeline for space debris KB question answering that utilizes out-of-domain and GPT-3 generated data for training, reducing overfitting.

Findings

Effective handling of complex space debris queries

Reduced data requirements through semi-synthetic data

Improved accuracy with pipeline decomposition

Abstract

Space agencies execute complex satellite operations that need to be supported by the technical knowledge contained in their extensive information systems. Knowledge bases (KB) are an effective way of storing and accessing such information at scale. In this work we present a system, developed for the European Space Agency (ESA), that can answer complex natural language queries, to support engineers in accessing the information contained in a KB that models the orbital space debris environment. Our system is based on a pipeline which first generates a sequence of basic database operations, called a %program sketch, from a natural language question, then specializes the sketch into a concrete query program with mentions of entities, attributes and relations, and finally executes the program against the database. This pipeline decomposition approach enables us to train the system by leveraging out-of-domain data and semi-synthetic data generated by GPT-3, thus reducing overfitting and shortcut learning even with limited amount of in-domain training data. Our code can be found at \url{https://github.com/PaulDrm/DISCOSQA}.