Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

TL;DR

This paper introduces Retrial-Augmented Learning (RAL), a novel self-supervised, retrial-based framework for large language models that enhances domain-specific decision-making and knowledge generation without additional training.

Contribution

The paper proposes a retrial-based, reward-free learning framework that enables autonomous knowledge generation and improves decision-making in LLMs without retraining or extensive data.

Findings

Reduces hallucination in LLMs by validated knowledge generation

Improves decision-making performance in complex environments

Shows robustness and transferability in out-of-distribution tasks

Abstract

The lack of domain-specific data in the pre-training of Large Language Models (LLMs) severely limits LLM-based decision systems in specialized applications, while post-training a model in the scenarios requires significant computational resources. In this paper, we present Retrial-Augmented Learning (RAL), a reward-free self-supervised learning framework for LLMs that operates without model training. By developing Retrieval-Augmented Generation (RAG) into a module for organizing intermediate data, we realized a three-stage autonomous knowledge generation of proposing a hypothesis, validating the hypothesis, and generating the knowledge. The method is evaluated in the LLM-PySC2 environment, a representative decision-making platform that combines sufficient complexity with domain-specific knowledge requirements. Experiments demonstrate that the proposed method effectively reduces hallucination by generating and utilizing validated knowledge, and increases decision-making performance at an extremely low cost. Meanwhile, the approach exhibits potential in out-of-distribution(OOD) tasks, robustness, and transferability, making it a cost-friendly but effective solution for decision-making problems and autonomous knowledge generation.