Ask Good Questions for Large Language Models

TL;DR

This paper introduces the AGQ framework that combines an enhanced CEIRT model with large language models to generate guiding questions, improving user guidance and information retrieval in dialog systems.

Contribution

It presents a novel combination of CEIRT and LLMs for generating guiding questions, enhancing dialog system performance in understanding user knowledge.

Findings

Outperforms baseline methods in guiding questions generation

Enhances information retrieval efficiency in dialog systems

Improves user experience in knowledge assessment

Abstract

Recent advances in large language models (LLMs) have significantly improved the performance of dialog systems, yet current approaches often fail to provide accurate guidance of topic due to their inability to discern user confusion in related concepts. To address this, we introduce the Ask-Good-Question (AGQ) framework, which features an improved Concept-Enhanced Item Response Theory (CEIRT) model to better identify users' knowledge levels. Our contributions include applying the CEIRT model along with LLMs to directly generate guiding questions based on the inspiring text, greatly improving information retrieval efficiency during the question & answer process. Through comparisons with other baseline methods, our approach outperforms by significantly enhencing the users' information retrieval experiences.