NeMo-Inspector: A Visualization Tool for LLM Generation Analysis
Daria Gitman, Igor Gitman, Evelina Bakhturina

TL;DR
NeMo-Inspector is an open-source visualization tool that streamlines the analysis and cleaning of synthetic datasets for large language models, improving data quality and model performance.
Contribution
The paper introduces NeMo-Inspector, a novel tool that simplifies synthetic dataset analysis and error correction for LLM training.
Findings
Reduced low-quality samples in GSM-Plus dataset from 46.99% to 19.51%.
Improved accuracy of LLMs on MATH and GSM8K datasets after data cleaning.
Demonstrated effectiveness in real-world synthetic data analysis cases.
Abstract
Adapting Large Language Models (LLMs) to novel tasks and enhancing their overall capabilities often requires large, high-quality training datasets. Synthetic data, generated at scale, serves a valuable alternative when real-world data is scarce or difficult to obtain. However, ensuring the quality of synthetic datasets is challenging, as developers must manually inspect and refine numerous samples to identify errors and areas for improvement. This process is time-consuming and requires specialized tools. We introduce NeMo-Inspector, an open-source tool designed to simplify the analysis of synthetic datasets with integrated inference capabilities. We demonstrate its effectiveness through two real-world cases. Analysis and cleaning of the synthetically generated GSM-Plus dataset with NeMo-Inspector led to a significant decrease in low-quality samples from 46.99% to 19.51%. The tool also…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsNatural Language Processing Techniques
