Column: Polls get it right most of the time. Here is what you should be careful of

Without polls, you can't understand politics. With them, you can misunderstand a lot.

With the presidential nominations decided by both major parties, we have entered a season in which anxiety about the polls can become an obsession. This is especially true this year, as former President Trump holds a small but persistent lead over President Biden in most national polls and swing states.

This has led many Democrats to dig deeper into the polls in an often disingenuous search for what is wrong.

The reality is that polls continue to get election results right the vast majority of the time. It is also an indispensable tool for democracy, informing the population of a vast and diverse country what their fellow Americans believe.

At the same time, errors exist, often involving problems in data collection or problems in interpreting them.

This week, let's examine some examples and take a look at how the Los Angeles Times polls are faring this primary season.

Holocaust myth?

In December, The Economist published startling poll results: “One in five young Americans believes the Holocaust is a myth,” read the headline.

Fortunately for the country, although perhaps not for the record, it is the poll result that is perhaps legendary.

In January, the nonpartisan Pew Research Center set out to see if it could replicate the results. They couldn't. Pew asked the same question as the Economist poll, and found that the percentage of Americans ages 18 to 29 who said the Holocaust was a myth was not 20%, but 3%.

What's going on?

The problem isn't a bad pollster: YouGov, which conducts polls for The Economist, is among the country's most respected pollsters. But the methodology used by YouGov, known in the polling world as opt-in panels, can fall victim to fake respondents. Maybe that was the case here.

Panel polls are a way to solve a big problem that pollsters face: very few people these days will answer phone calls from unknown numbers, which makes conducting traditional phone polls very difficult and very expensive.

Instead of randomly dialing phone numbers, polling organizations can recruit thousands of people who agree to take surveys, usually for a small sum. In each survey, pollsters select people from a panel to form a sample representative of the total population.

However, some people join just to get the money, and then may rush to answer questions almost randomly. Previous Pew research has found that these bogus respondents often claim to belong to groups that are difficult to recruit, including young people and Latino voters.

Pollsters have found evidence of organized efforts to hack committees, sometimes involving “multiple registrations from people outside the United States,” Douglas Rivers, chief scientist at YouGov and a political science professor at Stanford University, wrote in an email. These may be efforts to support specific causes or candidates, or more often, schemes to make money by raising small amounts over and over again.

“We have a full set of procedures to vet these committee members,” Rivers wrote, adding that the company continues to analyze what happened regarding the Holocaust issue.

In polls of close elections, fake respondents who respond at random usually “more or less cancel each other out,” said Andrew Mercer, chief research methodologist at Pew.

“But for something so rare, like Holocaust denial, random responses will produce error that is all in one side. It will end up inflating the infection rate,” he said.

In previous research, for example, Pew found that 12% of survey respondents who said they were under 30 also claimed to have been licensed to operate a nuclear submarine.

The lesson here is an old one, popularized by the late astronomer Carl Sagan: “Extraordinary claims require extraordinary evidence.” If a survey result seems too good to be true, there's a good chance it's not.

Jump to conclusions

The second category of potential problems has less to do with the data than with the way people, especially us journalists, interpret it: drawing definitive conclusions from less-than-definitive numbers.

Consider the extent of the progress Republicans are making among black and Latino voters.

There's no doubt, as I've written previously, that Republicans made progress between 2016 and 2020, especially among Latino voters who already identified as conservative. There was also a smaller movement toward the Republican Party among black voters.

Will this trend continue? Some recent surveys, including the widely cited New York Times/Siena College poll, suggest it may have accelerated. This poll found that Biden is bleeding support among young black and Latino voters.

In a recent article that attracted a lot of attention, John Byrne Murdoch, chief data journalist at the Financial Times, combined data from several different types of polls to declare that “American politics is in the midst of a racial realignment.”

The response of many political scientists and other analysts was actually “not so fast.”

They point out that pre-election polls can tell you what likely voters are thinking today, but comparing them to past election returns is risky.

If actual results in 2024 track what New York Times/Siena polls are currently finding, “well, let's talk about racial realignment,” said John Sides, a political science professor at Vanderbilt University. But until then, “we will have to wait and see.”

How we did

Polls conducted by the Berkeley Institute of Government and the Los Angeles Times had a remarkably good year at predicting the election.

The last poll before this year's primary, for example, showed that Proposition 1, a $6.4 billion mental health bond measure backed by Gov. Gavin Newsom, had the support of 50% of likely voters.

As of Thursday morning, that's where the “yes” vote stood at roughly 50.2%, with nearly 90% of the state's votes counted.

The poll also correctly predicted that Democratic Rep. Adam B. Schiff of Burbank and Republican former Dodgers player Steve Garvey would be in the top two in the Senate primary, with Democratic Rep. Katie Porter of Irvine in third place.

In the poll conducted about a week before the elections, 9% of voters remained undecided. The poll found that among those who had made up their mind, Garvey received 30% of the vote, Schiff 27%, and Porter 21%.

The poll appeared to be very close to Garvey's number — with about 800,000 votes yet to be counted, he got 32%, which is within the poll's estimated margin of error of two percentage points in either direction. The poll slightly underestimated support for Schiff, who also has 32%, and overestimated support for Porter, who currently has 15%. That could mean the final group of undecided voters split in Schiff's favor.

This level of accuracy is not uncommon. In the 2022 midterm elections, for example, polls conducted by nonpartisan groups, universities, and media organizations were highly accurate.

There's a word of advice in all of this for people interested in politics, especially in a hotly contested election year: Don't get too fixated on any single poll, especially if it has a surprising result not seen anywhere else. Be skeptical about sweeping conclusions about events that are still unfolding. And even, or perhaps especially, when a poll shows your favorite candidate in decline, take it for what it is — not a revelation or a nefarious conspiracy, but a snapshot in time.