Evidence of Systematic Bias in 2008 Presidential Polling (preliminary report)

TL;DR

This study provides evidence of systematic bias in 2008 presidential polls, showing that some media outlets' polls tend to favor certain political leanings, highlighting the need for further investigation into polling accuracy.

Contribution

It offers the first comparative analysis indicating potential biases among major polling sources during the 2008 presidential race.

Findings

Network polls are significantly more left-leaning than Gallup and Rasmussen.

A tentative order of media bias from right to left: FOX, CNN, NBC, ABC, CBS.

Results align with informal perceptions of media political leanings.

Abstract

Political polls achieve their results by sampling a small number of potential voters rather than the population as a whole. This leads to sampling error which most polling agencies dutifully report. But factors such as nonrepresentative samples, question wording and nonresponse can produce non-sampling errors. While pollsters are aware of such errors, they are difficult to quantify and seldom reported. When a polling agency, whether by intention or not, produces results with non-sampling errors that systematically favor one candidate over another, then that agency's poll is biased. We analyzed polling data for the (on-going) 2008 Presidential race, and though our methods do not allow us to identify which agencies' polls are biased, they do provide significant evidence that some agencies' polls are. We compared polls produced by major television networks with those produced by Gallup and Rasmussen. We found that, taken as a whole, polls produced by the networks were significantly to the left of those produced by Gallup and Rasmussen. We used the available data to provide a tentative ordering of the major television networks' polls from right to left. Our order was: FOX, CNN, NBC (which partners with the Wall Street Journal), ABC (which partners with the Washington Post), CBS (which partners with the New York Times). These results appear to comport well with the informal perceptions of the political leanings of these agencies. Our findings are preliminary, but they make a case for further research into the causes of and remedies for polling bias.