Let Me Teach You: Pedagogical Foundations of Feedback for Language Models

TL;DR

This paper introduces FELT, a pedagogically grounded framework for natural language feedback in large language models, aiming to systematize and improve feedback design based on learning sciences.

Contribution

It proposes a novel feedback framework and taxonomy for LLMs inspired by pedagogy, addressing the arbitrariness of current NLF methods and guiding future research.

Findings

Provides a comprehensive feedback taxonomy for LLMs

Offers a systematic mapping of feedback characteristics

Introduces new research directions in NLF

Abstract

Natural Language Feedback (NLF) is an increasingly popular mechanism for aligning Large Language Models (LLMs) to human preferences. Despite the diversity of the information it can convey, NLF methods are often hand-designed and arbitrary, with little systematic grounding. At the same time, research in learning sciences has long established several effective feedback models. In this opinion piece, we compile ideas from pedagogy to introduce FELT, a feedback framework for LLMs that outlines various characteristics of the feedback space, and a feedback content taxonomy based on these variables, providing a general mapping of the feedback space. In addition to streamlining NLF designs, FELT also brings out new, unexplored directions for research in NLF. We make our taxonomy available to the community, providing guides and examples for mapping our categorizations to future research.