Legal and Political Stance Detection of SCOTUS Language

TL;DR

This paper uses automated stance detection on Supreme Court documents to analyze the political and legal orientations of justices, revealing correlations with public opinion and introducing a new dataset for legal stance detection.

Contribution

It introduces a novel approach to measure judicial ideology through language analysis and presents a new dataset for legal stance detection with competitive results.

Findings

Justices more responsive to public opinion express their ideology during oral arguments.

Language-based ideology metrics correlate with existing social scientific measures.

Proposed legal stance detection dataset achieves competitive performance.

Abstract

We analyze publicly available US Supreme Court documents using automated stance detection. In the first phase of our work, we investigate the extent to which the Court's public-facing language is political. We propose and calculate two distinct ideology metrics of SCOTUS justices using oral argument transcripts. We then compare these language-based metrics to existing social scientific measures of the ideology of the Supreme Court and the public. Through this cross-disciplinary analysis, we find that justices who are more responsive to public opinion tend to express their ideology during oral arguments. This observation provides a new kind of evidence in favor of the attitudinal change hypothesis of Supreme Court justice behavior. As a natural extension of this political stance detection, we propose the more specialized task of legal stance detection with our new dataset SC-stance, which matches written opinions to legal questions. We find competitive performance on this dataset using language adapters trained on legal documents.