Knoll: Creating a Knowledge Ecosystem for Large Language Models

TL;DR

Knoll is a software infrastructure enabling users to create, curate, and integrate custom knowledge modules into large language models, enhancing their access to personalized and up-to-date information for diverse tasks.

Contribution

The paper introduces Knoll, a novel system that allows end-users to build and share knowledge modules for LLMs, expanding their informational scope beyond pre-trained data.

Findings

Using Knoll improves response quality in LLM interactions.

Over 200 users employed Knoll for various personalized tasks.

Knoll effectively integrates user-curated knowledge into LLM outputs.

Abstract

Large language models are designed to encode general purpose knowledge about the world from Internet data. Yet, a wealth of information falls outside this scope -- ranging from personal preferences to organizational policies, from community-specific advice to up-to-date news -- that users want models to access but remains unavailable. In this paper, we propose a knowledge ecosystem in which end-users can create, curate, and configure custom knowledge modules that are utilized by language models, such as ChatGPT and Claude. To support this vision, we introduce Knoll, a software infrastructure that allows users to make modules by clipping content from the web or authoring shared documents on Google Docs and GitHub, add modules that others have made, and rely on the system to insert relevant knowledge when interacting with an LLM. We conduct a public deployment of Knoll reaching over 200 users who employed the system for a diverse set of tasks including personalized recommendations, advice-seeking, and writing assistance. In our evaluation, we validate that using Knoll improves the quality of generated responses.