Assessing RAG and HyDE on 1B vs. 4B-Parameter Gemma LLMs for Personal Assistants Integretion

TL;DR

This paper evaluates RAG and HyDE augmentation strategies on 1B and 4B parameter Gemma LLMs for privacy-focused personal assistants, showing RAG improves latency and factual accuracy, while HyDE enhances relevance at higher computational cost.

Contribution

It provides a comparative analysis of RAG and HyDE on small-scale LLMs, highlighting RAG's suitability for resource-constrained personal assistant applications.

Findings

RAG reduces latency by up to 17% and eliminates hallucinations.

HyDE improves semantic relevance but increases response time and hallucination risk.

Scaling from 1B to 4B models offers marginal throughput gains but amplifies HyDE's overhead.

Abstract

Resource efficiency is a critical barrier to deploying large language models (LLMs) in edge and privacy-sensitive applications. This study evaluates the efficacy of two augmentation strategies--Retrieval-Augmented Generation (RAG) and Hypothetical Document Embeddings (HyDE)--on compact Gemma LLMs of 1 billion and 4 billion parameters, within the context of a privacy-first personal assistant. We implement short-term memory via MongoDB and long-term semantic storage via Qdrant, orchestrated through FastAPI and LangChain, and expose the system through a React.js frontend. Across both model scales, RAG consistently reduces latency by up to 17\% and eliminates factual hallucinations when responding to user-specific and domain-specific queries. HyDE, by contrast, enhances semantic relevance--particularly for complex physics prompts--but incurs a 25--40\% increase in response time and a non-negligible hallucination rate in personal-data retrieval. Comparing 1 B to 4 B models, we observe that scaling yields marginal throughput gains for baseline and RAG pipelines, but magnifies HyDE's computational overhead and variability. Our findings position RAG as the pragmatic choice for on-device personal assistants powered by small-scale LLMs.