Ignore the Polls

Or at least the current horse race polls

Now that we have your attention with what seems a man-bites-dog headline, we need to make an important distinction – between public opinion research and political polling.

Our clients hire us to conduct public opinion research that we use to provide them with data-driven counsel that helps them achieve their strategic objectives. Political campaigns and politicians use public opinion polling in much the same way – and we think that’s a good thing. In our representative democracy, it’s one way the will of the people can be reflected in policymaking – and even in campaigning – by asking people what matters most to them and evaluating their preferences for different potential solutions. 

On the other hand, the political polling you regularly see in the news media is a misapplication of the tools and techniques used to conduct public opinion research. We explain our perspective in the following, and we hope it will help you evaluate the political polling you see. 

Polls are not predictive. 

Polls are lagging indicators – the opposite of a predictive tool. At best, they reflect what public opinion is at a particular moment in time and not what it will be. Unfortunately, many often misjudge polls as being “right or wrong,” which reveals the flawed expectation that they are predictive.  

Polling can’t tell us about turnout. 

For any poll to be accurate and informative, the sample of people interviewed needs to be representative of the population in question. Whether it is voting age population, registered voters or people who vote regularly in presidential-year elections, the sample needs to be representative of the population based on known characteristics such as geography, gender, age, education and voting history. Vote likelihood is not a knowable characteristic. This is a critical point. Pollsters do their best to estimate voter turnout based on their own witch’s brew of data, questions and calculations – you’ll see this referred to as a “proprietary model” – but turnout is notoriously difficult to gauge because it requires sampling a population that does not yet exist (i.e., people who have voted in the upcoming election). Even likely voters are just that – likely, not definite. 

In 2016, Democratic pollsters underestimated the turnout of non-college educated white voters, which fouled up Democrats’ get-out-the-vote strategies. Ironically, the shoe was on the other foot in 2020 when Republican pollsters underestimated overall turnout. 

State polling matters more. 

National polls would be more telling if the nationwide popular vote determined who becomes president, but that’s not how our system works. The electoral college system means we have 50 separate presidential elections. Forty-three states reliably vote for one party’s nominee or the other. That leaves 93 electoral votes across seven states to decide who will win the presidency in 2024. If you’re going to watch anything, watch the polling in those seven states (AZ, GA, MI, NV, PA, WI and NC). That said, these are swing or toss-up states for a reason. Any advantage for either candidate likely will be on a knife’s edge. As if on cue, The Cook Political Report recently released just such a poll based on 3,969 likely voters distributed proportionate to the population of these seven states. 

Read beyond the headline. 

The presidential “horserace” question gets all the attention from the media, but quality political polls ask a lot of good questions that don’t get much attention. What issues matter most to voters? Who do voters think will do a better job handling different issue areas? Which candidate has preferable personality traits? Even these questions leave us wanting to know more, such as why people hold the opinions they do. That’s why it is so important to pair quantitative polls with a qualitative method such as focus groups. Focus groups may not be representative, but they reveal insights even the best poll cannot. So, keep an eye out for people such as Sarah Longwell, who regularly conducts focus groups and shares her findings via The Bulwark.    

Mind the sample sizes. 

Most polling coverage reports results based on the total sample, typically 1,000 interviews for a national poll, oftentimes fewer for state-level polls, depending on the size of the state (an intuitive, but technically inaccurate, approach we won’t get into here).  Every poll will have the ability to look at different audience segments (subgroups of that total sample) based on demographics, attitudes or behavior.  These subgroups are smaller sample sizes by definition, and with that comes less stability (greater variability) in the results. Beware the tendency to overinterpret a provocative finding. President Biden underperforming among Black voters or overperforming among white, non-college voters are two good examples of findings that should be taken with a grain of salt, unless the survey had large oversamples of each of those groups or the whole poll was conducted with either of those very specific audiences. Otherwise, the findings will be based on a very small number of survey respondents.

Relax and enjoy the spring and summer. Whether your candidate is up or down in the polls, paying too much attention to polls at this stage of on election year will leave you either overconfident or anxious. Most Americans – including voters and especially late deciders – are not paying attention this early in the presidential race, which means their views are not well-defined. So do yourself a favor: step away from the polling until after Labor Day and start your beach reading early.

If you’re not supposed to pay attention to polling, and you’re not ready to start your beach reading, what should you be paying attention to? You can find the answer to that question here.