# The ROOTS Search Tool: Data Transparency for LLMs

## TL;DR

The paper introduces the ROOTS Search Tool, an open-source search engine for the extensive ROOTS multilingual corpus, enhancing data transparency and governance for large language model training.

## Contribution

It presents the development and implementation of a comprehensive search tool for the ROOTS corpus, enabling detailed investigation and transparency of data used in LLM training.

## Key findings

- Largest searchable corpus with fuzzy and exact search capabilities
- Open-sourced tool available on Hugging Face Spaces
- Facilitates data transparency and governance in LLM training

## Abstract

ROOTS is a 1.6TB multilingual text corpus developed for the training of BLOOM, currently the largest language model explicitly accompanied by commensurate data governance efforts. In continuation of these efforts, we present the ROOTS Search Tool: a search engine over the entire ROOTS corpus offering both fuzzy and exact search capabilities. ROOTS is the largest corpus to date that can be investigated this way. The ROOTS Search Tool is open-sourced and available on Hugging Face Spaces. We describe our implementation and the possible use cases of our tool.