RAGnaroX: A Secure, Local-Hosted ChatOps Assistant Using Small Language Models

TL;DR

RAGnaroX is a secure, on-premise ChatOps assistant built with small language models, offering competitive accuracy and resource efficiency on commodity hardware, with full auditable control and modular architecture.

Contribution

It introduces RAGnaroX, a fully local, resource-efficient ChatOps assistant with modular design, implemented in Rust, and evaluated on multiple QA datasets.

Findings

Achieves 0.90 context precision on single-hop QA

Response time averages 2.5 seconds per request

Maintains competitive accuracy with resource efficiency

Abstract

This paper introduces RAGnaroX, a resource-efficient ChatOps assistant that operates entirely on commodity hardware. Unlike existing solutions that often rely on external providers such as Azure or OpenAI, RAGnaroX offers a fully auditable, on-premise stack implemented in Rust. Its architecture integrates modular data ingestion, hybrid retrieval, and function calling, enabling flexible yet secure deployment. Our evaluation focuses on the RAG pipeline, with benchmarks conducted on the SQuAD (single-hop QA), MultiHopRAG (multi-hop QA), and MLQA (cross-lingual QA) datasets. Results show that RAGnaroX achieves competitive accuracy while maintaining strong resource efficiency, for example, reaching 0.90 context precision on single-hop questions with an average response time of 2.5 seconds per request. A replication package containing the tool, the demonstration video (https://www.youtube.com/watch? v=cDxfuEbcoM4), and all supporting materials are available at https://github.com/genius-itea/RAGnaroX.git.