CAG: Chunked Augmented Generation for Google Chrome's Built-in Gemini Nano

TL;DR

CAG is a novel architecture that enables Google Chrome's Gemini Nano model to process large inputs efficiently within browser constraints by using intelligent input chunking, expanding AI capabilities directly in the browser.

Contribution

The paper introduces Chunked Augmented Generation (CAG), a new method for overcoming context window limitations in browser-based AI models like Gemini Nano.

Findings

Effective processing of large documents within Chrome

Maintains model performance with input chunking

Accessible AI capabilities directly in the browser

Abstract

We present Chunked Augmented Generation (CAG), an architecture specifically designed to overcome the context window limitations of Google Chrome's built-in Gemini Nano model. While Chrome's integration of Gemini Nano represents a significant advancement in bringing AI capabilities directly to the browser, its restricted context window poses challenges for processing large inputs. CAG addresses this limitation through intelligent input chunking and processing strategies, enabling efficient handling of extensive content while maintaining the model's performance within browser constraints. Our implementation demonstrates particular efficacy in processing large documents and datasets directly within Chrome, making sophisticated AI capabilities accessible through the browser without external API dependencies. Get started now at https://github.com/vivekVells/cag-js.