Development of REGAI: Rubric Enabled Generative Artificial Intelligence

TL;DR

This paper introduces REGAI, a novel AI technique that leverages rubrics to enhance large language models' evaluation capabilities, outperforming traditional methods and expanding potential application areas.

Contribution

REGAI is a new retrieval augmented generation approach that integrates rubrics, created manually or automatically, to improve LLM evaluation performance.

Findings

REGAI outperforms classical LLMs in evaluation tasks.

Rubrics enhance LLM performance in various applications.

The system can generate rubrics automatically.

Abstract

This paper presents and evaluates a new retrieval augmented generation (RAG) and large language model (LLM)-based artificial intelligence (AI) technique: rubric enabled generative artificial intelligence (REGAI). REGAI uses rubrics, which can be created manually or automatically by the system, to enhance the performance of LLMs for evaluation purposes. REGAI improves on the performance of both classical LLMs and RAG-based LLM techniques. This paper describes REGAI, presents data regarding its performance and discusses several possible application areas for the technology.