Nationwide polls are often of limited relevance, considering the unique structure of the US electoral system. To gain a better understanding of the upcoming presidential election, we need to focus on surveys conducted in the pivotal battlegrounds – the so-called swing states. After the missteps in previous elections, it’s hard to place too much confidence in these polls, as many rely on unrepresentative samples.
As we head toward the 2024 US presidential election, media large and small frequently fall into the trap of “horse race” journalism. Policy questions are rarely treated in depth, and the emphasis is often on the latest polls. One week they announce Kamala Harris as moving ahead, and the next, Donald Trump still has an edge. But how reliable are these polls?
In the United States, rather than being elected by direct popular vote, the president is chosen indirectly through the Electoral College, an institution inscribed in the country’s constitution. Each state is assigned a number of electors based in part on its population, but also on its number of senators. As a result, smaller states get a larger voice than their population would indicate.
One of the implications is that national election polls can be deceiving. In most states with established partisan majorities, the outcomes are predictable due to the winner-takes-all approach. This system awards all of a state’s electoral votes to the candidate who wins the popular vote in that state (with the exception of Maine and Nebraska, which use a proportional system). As a result, the most relevant polls are those conducted in “swing states”, where neither party holds a consistent advantage.
According to recent analyses, around ten states are expected to be in play for 2024. Based on recent trends, there are seven swing states to watch: Nevada, Arizona, Wisconsin, Michigan, Pennsylvania, North Carolina, and Georgia. In the 2016 and 2020 elections, victory margins in these states were razor-thin, often less than 1%.
With both Harris and Trump within striking distance of the 270 Electoral College votes needed to win the presidency, these swing states, with a combined 91 votes, will determine the outcome.
The 2016 and 2020 polling failures: flukes or systemic issues?
When the margins are so tight in these key states, accurately measuring voter intentions is an enormous challenge. In 2016, national polls correctly predicted Hillary Clinton’s popular-vote win – she had nearly 3 million more than Trump. However, they failed to foresee Trump’s Electorial College victories in critical states, which ultimately put him over the top.
The American Association for Public Opinion Research (AAPOR) pointed out several reasons for these errors, including underrepresentation of Republican voters, over-representation of college-educated voters (who tend to lean Democratic), and an underestimation of undecided voters who eventually voted for Trump or third-party candidates.
Despite efforts to fix these problems, other biases showed up in 2020. While graduate voters were not over-represented and undecideds were evenly split between Biden and Trump, the Covid-19 pandemic had made the pollsters’ task more complicated. AAPOR points out that the states with a higher proportion of Covid-19 cases were the ones with the highest polling errors. As a result, pollsters underestimated Trump’s vote share in key swing states and also overestimated Biden’s national lead, making the 2020 polls the least accurate in 40 years.
Despite these errors, Biden still triumphed, winning 4 percent more of the popular vote and taking home 306 electoral votes to Trump’s 232. Biden’s victories in the swing states of Arizona, Georgia, Michigan, Nevada, Pennsylvania, and Wisconsin make all the difference.
Polling errors and public distrust
Errors of this magnitude naturally increase the public’s scepticism of polling, especially among Republicans, who are already wary of establishment institutions. Contrary to initial assumptions, Trump voters didn’t hesitate to express their preferences in 2016 and 2020. However, they were less likely to participate in polls due in part to their distrust of mainstream institutions. As a result, working-class white voters – and their opinions – were underrepresented in many polls.
Pollsters also face technical challenges. Getting a respondent on the phone now requires calling hundreds of people, thanks to caller ID and call screening. Polls with smaller samples (fewer than 1,000 respondents) are less reliable. To deal with these hurdles, many pollsters are now using a mix of methods, including e-mail, online surveys, and robocalls.
Though cheaper, online surveys often draw voluntary participants who are compensated, which leads to issues of accuracy and representation. This growing reliance on online polling has contributed to a doubling of polling companies from 2000 to 2022, according to Pew Research Center.
Margin of error and identifying “likely” voters
The margin of error is a critical component of polling that is often misunderstood by the public and media. It typically falls between 3 and 4 percentage points, but for smaller demographic groups (for example, young people, white men, or Hispanics), it can be even higher. Media headlines, however, frequently imply a candidate is leading, even when the difference is within the margin of error. University of California, Berkeley researchers suggest that to ensure 95% accuracy, the margin of error should be closer to 6%.
However, the media sometimes amplify results, particularly in headlines, by implying that a candidate is ahead, even when the difference is within the margin of error. Moreover, researchers at the University of Berkeley have shown that to guarantee 95% accuracy, this margin should be increased to at least 6%. This means a candidate projected to receive 54% of the vote is likely, in reality, to secure anywhere between 48% and 60%, reflecting an actual margin of error of 12 percentage points.
Another significant challenge for pollsters is identifying likely voters. Only around two-thirds of citizens eligible to vote actually go to the polls. In 2016, turnout on the Democratic side was overestimated, giving the false impression that Clinton was a lock for victory. This likely caused some of her supporters to stay home, while Trump’s base showed up in force when polls suggested he was behind. Accurately predicting who will turn out to vote is crucial to polling accuracy.
Lessons from the 2022 midterms: A glimmer of hope for 2024?
Polling showed notable improvements during the 2022 midterm elections, with the results being the most accurate since 1998. Importantly, there was no significant bias toward either party. However, midterm elections operate differently than presidential elections, and the dynamics for 2024 may be very different. That said, many polling institutions have adapted since 2016: as of 2022, 61% of polling firms had changed their methods, such as refining sampling techniques and improving question wording. More than a third have changed their methods after 2020.
While these changes are positive, challenges remain, especially in predicting turnout and combating low response rates.
What good are polls, then?
At the end of the day, election polls offer snapshots – often imprecise – and can only provide general trends. Polling methods vary across firms, introducing biases that make it difficult to compare results.
Survey aggregators offer averages that might be more reliable than individual polls, but they still come with a degree of uncertainty. This is true for FiveThirtyEight, the well-known website founded by statistics guru Nate Silver. After ABC took over in 2023, Silver left, taking his forecasting model with him to his new platform, Silver Bulletin, which continues to attract significant media attention.
With the unpredictability of polls, political betting markets have become popular as polling alternatives. Platforms like Polymarket, which recently hired Silver, have multiplied rapidly. Some people, like Elon Musk, argue that markets provide better forecasts than traditional polls, though this claim is unproven. There are also concerns that these markets could be manipulated to sway public opinion.
While opinion polls aren’t the best tools for predicting elections – as this could be one of the closest in recent history – their value lies in gauging public opinion on key issues. However, even in this role, polls can still be biased, often influenced by how questions are phrased.
For example, in 2019 USA Today ran the headline “Poll: Half of Americans say Trump is victim of a ‘witch hunt’ as trust in Mueller erodes”. This was in reference to Special Counsel Robert Mueller’s investigation into Russian interference in the 2016 election. The question asked by the poll was:
“President Trump has called the special counsel’s investigation a ‘witch hunt’ and said he has been investigated more than previous presidents for political reasons. Do you agree?”
The problem with this wording is that it combined two different ideas: whether the investigation was a “witch hunt” and whether Trump had been unfairly targeted for political reasons. On top of that, the question lacked neutrality, presenting only his perspective.
Naturally, Trump used the result to his advantage, even though other polls from sources such as The Washington Post, CBS News, and NPR-PBS told a different story.
To use polling data wisely during this election, it’s crucial to recognize these limitations and pay attention to the fine print – details like the sample size, polling date, margin of error, and methodology. Additionally, consider the poll’s sponsors, who may only release results that align with their particular agenda.
Ultimately, the best way to interpret polling data is with caution, focusing on general trends rather than any single poll. And always remember, election outcomes can be full of surprises.
Jérôme Viala-Gaudefroy ne travaille pas, ne conseille pas, ne possède pas de parts, ne reçoit pas de fonds d'une organisation qui pourrait tirer profit de cet article, et n'a déclaré aucune autre affiliation que son organisme de recherche.
This article was originally published on The Conversation. Read the original article.
