Securing LLM-as-a-Service for Small Businesses: An Industry Case Study of a Distributed Chatbot Deployment Platform

TL;DR

This paper presents a cost-effective, secure, and easy-to-deploy platform enabling small businesses to utilize LLM-based chatbots with minimal technical expertise, addressing infrastructure, security, and operational challenges.

Contribution

It introduces an open-source, distributed, multi-tenant platform with integrated security defenses for deploying customized LLM chatbots in small business environments.

Findings

Demonstrates cost-efficient deployment on low-cost hardware

Shows effective mitigation of prompt injection attacks

Validates platform performance in real-world e-commerce setting

Abstract

Large Language Model (LLM)-based question-answering systems offer significant potential for automating customer support and internal knowledge access in small businesses, yet their practical deployment remains challenging due to infrastructure costs, engineering complexity, and security risks, particularly in retrieval-augmented generation (RAG)-based settings. This paper presents an industry case study of an open-source, multi-tenant platform that enables small businesses to deploy customised LLM-based support chatbots via a no-code workflow. The platform is built on distributed, lightweight k3s clusters spanning heterogeneous, low-cost machines and interconnected through an encrypted overlay network, enabling cost-efficient resource pooling while enforcing container-based isolation and per-tenant data access controls. In addition, the platform integrates practical, platform-level defences against prompt injection attacks in RAG-based chatbots, translating insights from recent prompt injection research into deployable security mechanisms without requiring model retraining or enterprise-scale infrastructure. We evaluate the proposed platform through a real-world e-commerce deployment, demonstrating that secure and efficient LLM-based chatbot services can be achieved under realistic cost, operational, and security constraints faced by small businesses.