Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios

TL;DR

This paper introduces a novel active learning framework with clustering to guide LLM generation, improving the diversity and robustness of safety scenario data for critical applications.

Contribution

It presents a new method combining active learning and clustering to generate more representative safety scenarios without prior distribution knowledge.

Findings

Generated 5.4K safety violation scenarios

Improved accuracy and F1 scores of models using the data

Enhanced diversity and robustness of safety data

Abstract

Ensuring robust safety measures across a wide range of scenarios is crucial for user-facing systems. While Large Language Models (LLMs) can generate valuable data for safety measures, they often exhibit distributional biases, focusing on common scenarios and neglecting rare but critical cases. This can undermine the effectiveness of safety protocols developed using such data. To address this, we propose a novel framework that integrates active learning with clustering to guide LLM generation, enhancing their representativeness and robustness in safety scenarios. We demonstrate the effectiveness of our approach by constructing a dataset of 5.4K potential safety violations through an iterative process involving LLM generation and an active learner model's feedback. Our results show that the proposed framework produces a more representative set of safety scenarios without requiring prior knowledge of the underlying data distribution. Additionally, data acquired through our method improves the accuracy and F1 score of both the active learner model as well models outside the scope of active learning process, highlighting its broad applicability.