Public opinion polls and surveys are attention getters, headline grabbers.

Reporters and editors love them. Sometimes they should learn to hate them … or at least to approach each new one with a healthy dose of skepticism.

Several polls this year have attracted media attention for suggesting that the public is backing away from the idea that climate change is happening or that action is needed.

In January, the Pew Research Center released a widely reported poll saying that the environment has taken a back seat to the economy, a first among serious pollsters. And subsequent polls by Gallup and Rasmussen have also seemingly chipped away at the notion that the public “believes in global warming” or favors steps to avoid it.

In an atmosphere characterized by a deep recession, few may be surprised that economic issues are trumping environmental ones, climate change especially, and that the public is reordering its priorities in response.

But whether or not public opinion is shifting in any fundamental way on climate change itself is unclear, and some say it’s unlikely. (See Washington Post-ABC News; Yale/George Mason; and Mellman Group, among others.)

Media’s Best Strategy: Healthy Skepticism

Of course, polling results on climate change issues remain voluminous and varied. Adversaries on all sides of the public policy debate are likely to find polling results to buttress their positions. The question that arises is just how good those results really are. And the phrasing, emphasis, and structure of the surveys themselves often make cross-comparisons and interpretations difficult, experts say.

The imperative for journalists is solid raised-eyebrow scrutiny.

“Do not be a victim of the wording of the latest polls,” Jon Krosnick, a polling expert and professor of communication and political science at Stanford University, said in a telephone interview with The Yale Forum. “Just because it happens, doesn’t mean it deserves coverage.” (Try telling that to your editor.)

Krosnick, who focuses on climate change issues, advocates a two-step process in evaluating any new opinion survey. First, look at the actual question put to respondents and make sure it is clear – that the wording isn’t “complicated with baggage.” Second, call a trained polling expert and ask for a second opinion on the methodologies behind the poll or survey.

Two examples from just this year offer object lessons in some of the pitfalls common to reporting on poll results. The Yale Forum asked Stanford’s Krosnick and former Gallup vice president David Moore, now a senior fellow with the Carsey Institute at the University of New Hampshire, to weigh in.

Examining Gallup’s March Poll

In March, a new Gallup poll claimed that 41 percent of Americans now believe that the seriousness of global warming is being “exaggerated” in the media, representing the “highest level of public skepticism about mainstream reporting” in more than a decade.

That result continues to be reported in the media, from Newsday and the San Francisco Chronicle to the Washington Examiner and USA Today. Some pieces use it to prop up arguments for and against more action on global warming; some cite it as one more statistic in a heated debate.

To Gallup Poll Editor Frank Newport, it is one piece of evidence that Al Gore’s campaign is losing. “It’s just not caught on,” Newport said in an interview with U.S. News & World Report. “They have failed.”

However, all citations of that particular Gallup poll rest on one underlying assumption: that it is statistically significant and worthy of the public’s attention.

But on closer analysis, what is the question really asking? And how much has opinion really moved over a significant amount of time?

It’s worth noting right away that, although Gallup highlighted that question with its headline – “Increased Number Think Global Warming Is Exaggerated” – in its March polling report, the number moved from 35 percent in 2008 to 41 percent in 2009. It was 38 percent in 2004, dropped to 30 percent in 2006, and now is back up.

Former Gallup pollster Moore says that longer trends are important – in fact, they may be the only things polls can really measure accurately. With this poll, we see a cycle up and down over five years, he points out, hardly big news.

Moreover, Krosnick calls it categorically a “bad question.” He says the question should have asked what percentage of news reporting constituted exaggeration. As stated – “Thinking about what is said in the news, is the seriousness of global warming exaggerated or correct/underestimated?” – it doesn’t assess whether or not Americans are reacting to some extremes in a more intense news environment or the bulk of the mainstream coverage.

And, Krosnick adds, it’s simply not a fair, straightforward way of assessing real opinion about global warming. (Note that the report’s headline doesn’t immediately make clear that this is a verdict on the press, not on the scientific phenomena.)

Respondents, Moore points out, basically still have the same views year to year on the poll as a whole. On questions of whether global warming “will pose a serious threat to you or your way of life in your lifetime,” it’s a margin-of-error wash: 40 percent in 2008, 38 percent in 2009. On the question of whether those surveyed “personally worry,” it was 66 percent in 2008, 60 percent in 2009. (And over time, it’s been as high as 72 percent in 2000, as low as 51 percent in 2004.)

“They may have kind of misled people by focusing on an issue that may not be fundamental,” Moore says of the “exaggeration” question. He added that if he were still doing the polling, “I would have a hard time writing a story” based on the total results presented.

Moore, who authored the recent book “The Opinion Makers: An Insider Exposes the Truth Behind the Polls,”has argued extensively that the public often does not have strong opinions on issues, and that polls serve to artificially “manufacture” a body of solid opinion on one side or another.

For his part, Krosnick said the survey research he has been involved with has shown no fundamental changes in the public’s view on climate change, though he acknowledges that the economy has been taking over when the public is asked to rank priorities.

Scrutinizing Rasmussen April Poll

In April, Rasmussen Reports released a new poll on climate change and packaged it with the following headline: “Only 34% Now Blame Humans for Global Warming.”

The year-to-year change seemed significant: 47 percent in 2008 to 34 percent this year.

But a closer look yields two potentially troubling facets of the poll itself. First, the voters who were surveyed were forced into a choice: either they believe global warming is caused by “human activity” or by “long term planetary trends.” “Some other reason” and “not sure” were also available as answers.

Yet what wasn’t available was “both.”

“More than 80 percent of Americans think that nature and man are responsible,” Krosnick said in the interview. “But when you ask just about man-made global warming, you get lower numbers.”

Moore also calls that forced choice “really flakey.” And he points to a subtle but possibly significant difference in the survey structure year-to-year: the 2008 and 2009 polls, both five questions long, differ in how they lead up to the big question.

Last year, the first question was about how the federal government was handling environment. In 2009, the first poll question was: “Which is more important, finding new sources of energy or reducing the amount of energy Americans now consume?”

Moore contends that putting the energy issue – whether it brings to mind oil drilling, or alternative forms of wind or solar – in the minds of those surveyed could have steered them on the subsequent question about human activity versus planetary trends. Of course, one could debate which way people were steered. But the fact remains that the polls are not the same, Moore said: “You primed them in a certain direction.”

And the point stands that it’s worth scrutinizing how the poll is set up and sequenced, especially when there are year-to-year comparisons.

Surprise! Recent Weather Can Affect Poll Results

Recent hot or cool weather … implications for climate polling results.

There is a daunting body of research about polling itself – about the psychological underpinnings and proper methods for conducting statistically valid surveys and polls. For journalists and other non-expert observers on tight deadlines, it may inevitably be somewhat a matter of faith that polls say what pollsters claim. Not something most journalists and their editors want to hear or acknowledge.

But just to show how shaky surveying opinion on climate change can be, look no further than a recent working paper by academics Patrick Egan and Megan Mullin. They studied the impact between the local weather conditions people experience and their views on global warming.

Their finding? To perhaps no one’s surprise, there is a significant relationship, at least for the less educated: “If the temperature recently has been warmer than usual, respondents with unformed beliefs will be more likely to express a judgment that the Earth is getting warmer.” One can imagine pollsters facing trouble during a particularly warm winter or chilly summer.

As polling analyst Nate Silver points out at his Five Thirty Eight blog, other factors also may figure into people’s opinions right now: a kind of mystical belief that a “pro-environment” President will magically solve the global warming issue; the global slowdown in industrial production, and its impact on greenhouse gases; and even the relatively cooler temperatures of 2008, particularly in North America.

Finally, to make matters more complex, there is continuing debate and uncertainty about whether the very term “global warming” is a problem for pollsters and politicians alike – whether it doesn’t truly capture the scale of the danger, or is too ideologically loaded … and whether “climate change” is any better … or any worse.

You’ve heard about the one in which ecoamerica.org has confidentially recommended to the Natural Resources Defense Council using the term “our deteriorating atmosphere,” rather than climate change or global warming, as the term of choice? (See related news note published with this update.) Just wait to see what happens if pollsters ever get around to trying that one out on the public. But don’t hold your breath.