Decomposed Prompting to Answer Questions on a Course Discussion Board

TL;DR

This paper presents a question-answering system that uses decomposed prompting and question classification to improve responses on a course discussion board, achieving 81% classification accuracy.

Contribution

It introduces a novel decomposed prompting approach combined with question classification to tailor answers, enhancing accuracy in an educational setting.

Findings

81% classification accuracy on question types

Effective answering of conceptual questions in machine learning

Analysis of failure modes in the system

Abstract

We propose and evaluate a question-answering system that uses decomposed prompting to classify and answer student questions on a course discussion board. Our system uses a large language model (LLM) to classify questions into one of four types: conceptual, homework, logistics, and not answerable. This enables us to employ a different strategy for answering questions that fall under different types. Using a variant of GPT-3, we achieve $81\%$ classification accuracy. We discuss our system's performance on answering conceptual questions from a machine learning course and various failure modes.