Comparing Developer and LLM Biases in Code Evaluation

TL;DR

This paper introduces TRACE, a framework for evaluating how well LLMs as code judges align with human preferences and reveals systematic biases in their assessments across various coding tasks.

Contribution

The paper presents TRACE, a novel framework for analyzing biases in LLM-based code evaluation and compares model judgments to human preferences across multiple modalities.

Findings

Best LLM judges underperform humans by 12-23%

TRACE identifies 35 significant bias sources in LLM judgments

Judges are biased towards longer explanations in chat-based coding

Abstract

As LLMs are increasingly used as judges in code applications, they should be evaluated in realistic interactive settings that capture partial context and ambiguous intent. We present TRACE (Tool for Rubric Analysis in Code Evaluation), a framework that evaluates LLM judges' ability to predict human preferences and automatically extracts rubric items to reveal systematic biases in how humans and models weigh each item. Across three modalities -- chat-based programming, IDE autocompletion, and instructed code editing -- we use TRACE to measure how well LLM judges align with developer preferences. Among 13 different models, the best judges underperform human annotators by 12-23%. TRACE identifies 35 significant sources of misalignment between humans and judges across interaction modalities, the majority of which correspond to existing software engineering code quality criteria. For example, in chat-based coding, judges are biased towards longer code explanations while humans prefer shorter ones. We find significant misalignment on the majority of existing code quality dimensions, showing alignment gaps between LLM judges and human preference in realistic coding applications.