Natural Language Interaction with Databases on Edge Devices in the Internet of Battlefield Things

TL;DR

This paper presents a workflow using edge-sized Large Language Models to enable natural language querying and summarization of databases in battlefield IoT environments, enhancing situational awareness and decision-making.

Contribution

It introduces a novel two-step approach employing LLMs for mapping natural language questions to database queries and summarizing results, relaxing exact match constraints for improved accuracy.

Findings

Llama 3.1 outperforms other models across metrics

19.4% increase in query accuracy by relaxing exact match constraints

Demonstrates feasibility of deploying LLMs on edge devices for battlefield data interaction

Abstract

The expansion of the Internet of Things (IoT) in the battlefield, Internet of Battlefield Things (IoBT), gives rise to new opportunities for enhancing situational awareness. To increase the potential of IoBT for situational awareness in critical decision making, the data from these devices must be processed into consumer-ready information objects, and made available to consumers on demand. To address this challenge we propose a workflow that makes use of natural language processing (NLP) to query a database technology and return a response in natural language. Our solution utilizes Large Language Models (LLMs) that are sized for edge devices to perform NLP as well as graphical databases which are well suited for dynamic connected networks which are pervasive in the IoBT. Our architecture employs LLMs for both mapping questions in natural language to Cypher database queries as well as to summarize the database output back to the user in natural language. We evaluate several medium sized LLMs for both of these tasks on a database representing publicly available data from the US Army's Multipurpose Sensing Area (MSA) at the Jornada Range in Las Cruces, NM. We observe that Llama 3.1 (8 billion parameters) outperforms the other models across all the considered metrics. Most importantly, we note that, unlike current methods, our two step approach allows the relaxation of the Exact Match (EM) requirement of the produced Cypher queries with ground truth code and, in this way, it achieves a 19.4% increase in accuracy. Our workflow lays the ground work for deploying LLMs on edge devices to enable natural language interactions with databases containing information objects for critical decision making.