# Navigating the Seas of AI: Effectiveness of Small Language Models on Edge Devices for Maritime Applications

## TL;DR

This paper examines how small language models on battery-powered edge devices can be used for maritime navigation tasks when internet is unavailable.

## Contribution

The study introduces a RAG-based approach to enhance SLM performance for maritime documentation queries on edge devices.

## Key findings

- Small language models can operate effectively on edge devices in offline maritime environments.
- Retrieval-augmented generation improves response quality by integrating external sailing direction data.
- Inference times and response quality of SLMs were evaluated across multiple models for maritime use cases.

## Abstract

This paper explores the feasibility of employing small language models (SLMs) on edge devices powered by batteries in environments with limited/no internet connectivity. SLMs in fact offer significant advantages in such scenarios due to their lower resource requirements with respect to large language models. The use case in this study is maritime navigation—in particular, the documentation on Sailing Directions (Enroutd) of the World Port Index (WPI) provided by the National Geospatial-Intelligence Agency (NGA), which provides information that cannot be shown graphically on nautical charts and is not readily available elsewhere. In this environment, response immediacy is not critical, as users have sufficient time to query information while navigating and planning activities, making edge devices ideal for running these models. On the contrary, the response quality is fundamental. For this reason, given the constrained knowledge of SLMs in maritime contexts, we investigate the use of the retrieval-augmented generation (RAG) methodology, integrating external information from sailing directions. A comparative analysis is presented to evaluate the performance of various state-of-the-art SLMs, focusing on response quality, the effectiveness of the RAG component, and inference times.