A Multi-Stage Workflow for the Review of Marketing Content with Reasoning Large Language Models

TL;DR

This paper introduces a multi-stage workflow using fine-tuned reasoning LLMs to automatically review marketing content for compliance, comparing different training strategies and reward functions to optimize performance.

Contribution

It presents a novel, knowledge-independent approach for compliance issue detection and evaluates various fine-tuning and training methods for reasoning LLMs.

Findings

Fine-tuned reasoning LLMs effectively identify compliance issues.

Different training strategies impact model performance.

Reward function choices significantly influence outcomes.

Abstract

Reasoning Large Language Models (LLMs) have shown promising results when tasked with solving complex problems. In this paper, we propose and evaluate a multi-stage workflow that leverages the capabilities of fine-tuned reasoning LLMs to assist in the review process of marketing content, making sure they comply with a given list of requirements. The contributions of this paper are the following: (i) we present a novel approach -- that does not rely on any external knowledge representation -- for the automatic identification of compliance issues in textual content; (ii) compare the effectiveness of different fine-tuning strategies like Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) in training models to solve this problem; (iii) we evaluate the effectiveness of training small LLMs to generate reasoning tokens before providing their final response; (iv) we evaluate how the choice and combinations of different reward functions affects the performance of a model trained with GRPO.