Beyond Text-to-SQL for IoT Defense: A Comprehensive Framework for Querying and Classifying IoT Threats

TL;DR

This paper introduces a new IoT-specific text-to-SQL dataset and demonstrates how joint training can enhance SQL query accuracy and enable data classification, addressing the challenge of inferring new information from returned data.

Contribution

The paper presents a novel IoT-focused text-to-SQL dataset with complex queries and a two-stage processing approach for data classification, advancing the capabilities of natural language interfaces in IoT security.

Findings

Joint training improves text-to-SQL performance.

Large language models struggle with inferring new data information.

Dataset enables complex domain-specific reasoning in IoT contexts.

Abstract

Recognizing the promise of natural language interfaces to databases, prior studies have emphasized the development of text-to-SQL systems. While substantial progress has been made in this field, existing research has concentrated on generating SQL statements from text queries. The broader challenge, however, lies in inferring new information about the returned data. Our research makes two major contributions to address this gap. First, we introduce a novel Internet-of-Things (IoT) text-to-SQL dataset comprising 10,985 text-SQL pairs and 239,398 rows of network traffic activity. The dataset contains additional query types limited in prior text-to-SQL datasets, notably temporal-related queries. Our dataset is sourced from a smart building's IoT ecosystem exploring sensor read and network traffic data. Second, our dataset allows two-stage processing, where the returned data (network traffic) from a generated SQL can be categorized as malicious or not. Our results show that joint training to query and infer information about the data can improve overall text-to-SQL performance, nearly matching substantially larger models. We also show that current large language models (e.g., GPT3.5) struggle to infer new information about returned data, thus our dataset provides a novel test bed for integrating complex domain-specific reasoning into LLMs.