SenseAI: A Human-in-the-Loop Dataset for RLHF-Aligned Financial Sentiment Reasoning

TL;DR

SenseAI is a comprehensive human-in-the-loop dataset for financial sentiment reasoning that captures reasoning processes, confidence, and real-world outcomes to improve LLM alignment and understanding.

Contribution

The paper introduces SenseAI, a novel dataset with reasoning chains, confidence scores, and correction signals, tailored for RLHF-aligned financial AI model training.

Findings

Identifies systematic patterns in model errors, including Latent Reasoning Drift.

Reveals confidence miscalibration and forward projection tendencies in models.

Supports targeted model improvement through structured HITL data.

Abstract

We introduce SenseAI, a human-in-the-loop (HITL) validated financial sentiment dataset designed to capture not only model outputs but the full reasoning process behind them. Unlike existing resources, SenseAI incorporates reasoning chains, confidence scores, human correction signals, and real-world market outcomes, providing a structure aligned with Reinforcement Learning from Human Feedback (RLHF) paradigms. The dataset consists of 1,439 labelled data points across 40 US-listed equities and 13 financial data categories, enabling direct integration into modern LLM fine-tuning pipelines. Through analysis, we identify several systematic patterns in model behavior, including a novel failure mode we term Latent Reasoning Drift, where models introduce information not grounded in the input, as well as consistent confidence miscalibration and forward projection tendencies. These findings suggest that LLM errors in financial reasoning are not random but occur within a predictable and correctable regime, supporting the use of structured HITL data for targeted model improvement. We discuss implications for financial AI systems and highlight opportunities for applying SenseAI in model evaluation and alignment.