LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation

TL;DR

LLM BiasScope is an open-source web platform enabling real-time, side-by-side comparison and bias analysis of multiple large language models through automated detection, visualization, and detailed bias profiling.

Contribution

It introduces a novel real-time bias detection and comparison system supporting multiple LLM providers with interactive visualizations and export features.

Findings

Supports multiple LLM providers including Google Gemini and Meta Llama.

Provides real-time bias detection with visual summaries and detailed bias type breakdowns.

Enables side-by-side comparison of model outputs with bias analysis.

Abstract

As large language models (LLMs) are deployed widely, detecting and understanding bias in their outputs is critical. We present LLM BiasScope, a web application for side-by-side comparison of LLM outputs with real-time bias analysis. The system supports multiple providers (Google Gemini, DeepSeek, MiniMax, Mistral, Meituan, Meta Llama) and enables researchers and practitioners to compare models on the same prompts while analyzing bias patterns. LLM BiasScope uses a two-stage bias detection pipeline: sentence-level bias detection followed by bias type classification for biased sentences. The analysis runs automatically on both user prompts and model responses, providing statistics, visualizations, and detailed breakdowns of bias types. The interface displays two models side-by-side with synchronized streaming responses, per-model bias summaries, and a comparison view highlighting differences in bias distributions. The system is built on Next.js with React, integrates Hugging Face inference endpoints for bias detection, and uses the Vercel AI SDK for multi-provider LLM access. Features include real-time streaming, export to JSON/PDF, and interactive visualizations (bar charts, radar charts) for bias analysis. LLM BiasScope is available as an open-source web application, providing a practical tool for bias evaluation and comparative analysis of LLM behaviour.