It is time to stop quivering in our boots in pointless fear of the future and just roll up our sleeves and build it.
- Ray Pierrehumbert

Thursday, February 11, 2010

Truth, Consensus, and IPCC

As scientists, we believe that certain propositions are demonstrably and objectively true, and that our objective is to determine which ones they are. Given that we are in the public employ, the public has some say about which matters we are paid to investigate, and we try to align our own interests with those of the public.

Before people were going around wishing climate scientists dead, we had rather interesting lives, as there was a social motivation to pursue an incredibly interesting set of questions about how the world is put together. Not every intellectual style is happy with our pursuits; evidence tends to be messy, experiments equivocal, models approximate. On the other hand, the contacts with other fields are varied, and contacts with other countries inevitable. Unfortunately, we started to come across some rather bad news.

The beliefs of the best-informed people held together. Evidence that didn't align was almost invariably found faulty, as the picture was refined. By 1979, the outlines of the present consensus had emerged among the most accomplished physical scientists, and rather quickly became the basis for further research.

As time progressed, the news became bad enough that we had to start drawing the public's attention to it. Accordingly, an international body, the IPCC, was formed to collate and represent that news to decision makers and the public. There is evidence that IPCC was deliberately constituted so as to understate risks, largely under the influence fo the Reagan administration.
The Reagan administration wanted to forestall pronouncements by self-appointed committees of scientists, fearing they would be 'alarmist.' Conservatives promoted the IPCC's clumsy structure, which consisted of representatives appointed by every government in the world and required to consult all the thousands of experts in repeated rounds of report-drafting in order to reach a consensus. Despite these impediments the IPCC has issued unequivocal statements on the urgent need to act.
The IPCC summaries of the "consensus" were intended to report the state of science, not to define it. The "consensus" was and is just the spectrum of opinion held by working scientists. Some questions often raised by the public are considered "settled" in the community, while others are not. (Nobody claims "the science is settled" as a whole. This is a straw man.) The purpose of IPCC is not to do research but to review it and report upon it.

Nobody "worked for" the IPCC. (A few WMO employees spend most time on IPCC business. Pachauri himself draws no salary from IPCC.)

The climate science was reported by the first working group, and other issues (respectively impacts, and strategies) by the second and third working groups. Was this a good idea? I was not alone in being unimpressed by the last WG 2 imapcts assessment. Two far more accomplished and prominent U of Texas scientists than myself from opposite ends of the "main hump" (one an ecologist specializing in climate impacts and one a climatologist who is also a member of the American Association of Petroleum Geologists) have expressed disdain for the last WG 2 assessment to my own knowledge. A recent article in the Boulder Daily Camera takes a similar position (emphasis added):
No errors have surfaced in the first and most well-known of the reports, which said the physics of a warming atmosphere and rising seas is man-made and incontrovertible. So far, four mistakes have been discovered in the second report, which attempts to translate what global warming might mean to daily lives around the world. "A lot of stuff in there was just not very good," said Kevin Trenberth, head of climate analysis at the National Center for Atmospheric Research in Boulder and a lead author of the first report. "A chronic problem is that on the whole area of impacts, getting into the realm of social science, it is a softer science. The facts are not as good." ... No errors have surfaced in the first and most well-known of the reports, which said the physics of a warming atmosphere and rising seas is man-made and incontrovertible. So far, four mistakes have been discovered in the second report, which attempts to translate what global warming might mean to daily lives around the world. "A lot of stuff in there was just not very good," said Kevin Trenberth, head of climate analysis at the National Center for Atmospheric Research in Boulder and a lead author of the first report. "A chronic problem is that on the whole area of impacts, getting into the realm of social science, it is a softer science. The facts are not as good."
Stoat piles on, and is quoted in the Guardian as saying
"While some of the WG2 is fine, it is clear that some sections have been edited by people who should not have been trusted with the job.It should be done more on merit. At the very least, get someone competent to review the edit comments for their sections."
In short, there really is an objective system that physical science can be done on, and there is enough geophysics for a coherent set of positions to emerge on it. We can call this the consensus. The IPCC WG1 report attempts to report this consensus. It does not define the consensus. Conceivably IPCC WG 1 could misrepresent the consensus. Science attempts to find truth, and IPCC attempts to match science.

What of groups 2 and 3? Here one could argue that there is no consensus. There are too many aspects of the impacts question, and too many approaches to the mitigation/adaptation question. It is important for policy makers to understand the spread of professional opinion on these matters. Many people believe that the most recent working group 2 report failed to do so.

I strongly hope that the absurd hooting over the Himalaya glaciers error is not enough to hound Pachauri out of IPCC leadership. Whether he has done a good job or not, we need to defend people against the strategy of taking minor errors and missteps, and using them to destroy the careers and reputations of good, smart and decent people.

We cannot function with only politicians in charge. Somebody has to represent science, and scientists often have strong personalities of the sort politicians learn early to suppress, and while generally ethical, perhaps more of a detailed sense of the laws of nature than those of humans.

That doesn't mean we shouldn't revisit the role of IPCC and its achievements and failures. WG 2 seems to have been overrun by people with political agendas. The bad sort of committee thinking is painfully obvious to anyone who reads it. The WG 1 report in AR 4 continues to uphold the fine standards of the previous cycles. On the whole, the geosciences community deserves congratulations and respect for its efforts.

Anyway, ironically behind the paywall, Nature has some interesting ideas on the future of IPCC. I'm sympathetic to (of all people) John Christy, at least on process:
voluminous printed reports, issued every six years by government-nominated authors, cannot accommodate the rapid and chaotic development of scientific information today. An idea we pitched a few years ago that is now worth reviving was to establish a living, 'Wikipedia-IPCC'. Groups of four to eight lead authors, chosen by learned societies, would serve in rotating, overlapping three-year terms to manage sections organized by science and policy questions (similar to the Fourth Assessment Report). The authors would strike a balance between the free-for-all of true science and the need for summary statements.

Controversies would be refereed by the lead authors, but with input from all sides in the text, with links to original documents and data. The result would be more useful than occasional big books and would be a more honest representation of what our fledgling science can offer. Defining and following rules for this idea would be agonizing, but would provide greater openness.


Image of Ragendra Pachauri by evstafiev via Wikipedia

16 comments:

Marion Delgado said...

As I think you know by now, this is the source of my anger and determination to fight to the bitter end, really:

"we need to defend people against the strategy of taking minor errors and missteps, and using them to destroy the careers and reputations of good, smart and decent people."

I just re-read Ibsen's "An Enemy of the People." In some ways it's almost Randite, with his diatribes against the public and the tyranny of the majority. The doctor in question is trusting the lab implicitly, and he's not a nice person, and he doesn't pick his battles, and is a mess. But the things that happen to him, the rollercoaster from acceptance to ruin, are not what should happen when someone does his job and the public welfare is at stake.

John F. Pittman said...

Michael, I am reminded of what a teacher tells the parents of children in her class at the start of the year "If you agree not to believe everything your kids tell you about me, I will agree not to believe everything your kids tell me about you."

Though you may like WG1, there is room to consider that others may have a different conclusion for different reasons. I am a frequentist. When in WG1 it states that the IPCC recognizes that the models (all) have some problems, it assumes within the Bayesian a priori that they average out to the right answer, there are several problems. One is I have concerns for the potential bias (say 1C)that could be there. Also, when only accents the catastrophic past this Bayesian inferencing, such as the precautionary principle, in a way, one is challenging the framework on which the WG1 conclusions were based. It seems disingenous that those who claim to be backing the IPCC violate its methodology, and throw stones at those who note it. The tag applied to the persons in such a case is being anti-scientific and anti-IPCC. One cannot have it both ways. There are other examples.

Arthur said...

Marion - I was just thinking "Enemy of the People" reading Michael's article here... Really, has anybody ever figured out how to resolve that one issue of how to get the public to respect the bearer of bad messages? Cassandra was right, after all...

But this wikipedia-IPCC idea sounds perfect, is there perhaps some way we can help get it rolling before any official IPCC sanction?

Dion said...

Thanks for another insightful post.

You have a section repeated in the Denver newspaper quote, a copy/paste error I think.

Steve Bloom said...

I think Christy has something rather different in mind. It involves featuring his crap results.

Michael Tobis said...

Steve, yeah, I don't doubt that Christy wants to start over to give more weight to the sort of thing Christy likes.

That doesn't make it a bad idea intrinsically. The rules of who gets to do what on a site like that are crucial, as well as the initial recruitment.

RichieRIch said...

Whilst it may not be reasonable to call for Pachauri to resign on the basis of the errors that have so far been picked up in the IPCC reports, it may well be (more) reasonable to call for his resignation on the basis of his (in my view, inadequate) response to the detection of those errors.

Marion Delgado said...

Arthur and Michael I think it's a terrible idea, inherently - Wikipedia is really no model for investigative science for one thing - but even if I'm completely wrong, I see no possibility of it turning out well, at all. It'd end up as blogscience writ large, with no standards and money buying the largest volume of results.

If anything, we have to start protecting people with institutions - with everything we can restore or shore up or create. The Spanish Anarchists did just fine until they got caught between the Axis and Stalin's heavy handed support of the Republic, and then it didn't work so well. I think public interest science is more at that phase now.

Also, my other broken record: WG2 and WG3 were their own processes. Why assume that a small portion of errors means that for the purposes for which they were constituted they're flawed, vs. that being a likely outcome of being inclusive. The less hard science your group is, the more sources you can include, and these are public policy issues! The sheer volume of work means there will be mistakes. Maybe you just need more rounds of editing and scrutiny?

EliRabett said...

think scientology

Michael Tobis said...

Eli, you're being cryptic again.

Marion, it depends on what part of "wikipedia" you mean.

I think we're mostly talking about continuous publication and updates, rather than the awkward 6 year cycle which is now having an awkward impact on scientific process.

The ways in which statements would get included as consensus vs minority opinion are crucial. It's a long way from a vague idea to an official implementation.

An alternative would be a separately funded effort along these lines. As always, there is a tradeoff between advocacy and authority.

ed_davies said...

Dion: You have a section repeated in the Denver newspaper quote, a copy/paste error I think.

Actually, the repetition is in the Boulder newspaper quote. Big nits have smaller nits upon their backs to bite them, or something. Still, it'd avoid a minor distraction if the original article was fixed.

More substantially, I'd suggest that the Copenhagen Diagnosis is perhaps a better model than some sort of wiki, perhaps with a series of reports on various aspects of climate science as the consensus firms up in various areas. I wonder if part of the problem with WG2 is that it's simply premature due to the lack of overall consensus on impacts and that less ambitious efforts could be more reliable.

Steve Bloom said...

Solar astronomers (among others) are using Living Reviews, which may provide a suitable model.

Kooiti MASUDA said...

I have several loosely related things to say. (Perhaps I write more on my blog later.)

IPCC. I think that IPCC is a good structure to assemble knowledge about global climate change, but not so much about regional climate change. It must deal with reginal climate because impacts on ecosystems and on human societies occur locally. But, to know that the impact is likely to be so large to warrant mitigation measures, we do not need to examine every region. Inference from adequate samples would suffice.

Of couse, people in every region must adapt to changing climate with renewable resources rather than fossil fuels. Surely we need regional environmental assessments. But I do not think IPCC will be a primary organization to coordinate them worldwide.

Pachauri. I do not think Pachauri breached ethics, and I hope that he can continue working. But unfortunately he faces a charge of conflict of interest difficult to defend. From the Science interview we can know that Pachauri is faithful to TERI and that TERI is a non-profit organization. But his argument so far does not clear the doubts that Pachauri compromises science to the benefit of TERI. Hasnain, the glaciologist whose speculation ended up in IPCC AR4 WG2, works for TERI to get (largely foreign) funding for glacier research. Some allege that Pachauri intentionally overhyped the glacier melt in IPCC reports in order to get funding to TERI. I think the charge ridiculous. Also I can point out that saying "science is not settled" may be more effective in getting funds. But I can say that here is a potential conflict of interests.

(By the way, I remember, S.-I. Akasofu, former director of the International Arctic Research Center at the University of Alaska (an expert of aurora borealis), took the stance of "science is not settled". More precisely, he insisted that AGW cannot explain most of the recent environmental changes in the Arctic. After retirement, he is now a semi-denier of AGW, saying that just one sixth of current warming is explained by CO2. Trying to be sympathetic to him, I suspect that he was so much immersed in propaganda by a crowd of lobbyists during his own lobbying for science funds.)

Wikipedia. I find curious similarity between IPCC and Wikipedia that both aviod their own "original research" and seek for "reliable sources". (Of course, the meaning of these terms are not exactly the same between these communities.) These may be necessary codes for making heterogeneous crowd to produce something with good quality. But substantial fraction of people are not comfortable working with these constraints.

A flaw of Wikipedia that it's too quick. Anyone (except those who are banned) can change its official version any time. This characteristics is natural since its name came from "wiki-wiki" meaning "quick" in Hawaiian. It should be mixed with an oppsite indigredient. I do not know the word in Hawaiian. I happen to know that a word "pole-pole" in Swahili (since it's not English, both e's are pronounced) means "slowly" or "relaxed". (I have no other knowledge about that language. e.g. I do not know what a single "pole" may mean.)

Michael Tobis said...

Lovely... Polehpedia...

Kooiti MASUDA said...

Another fragment of my thoughts:

Science-society interaction.
We hear about the "deficit model" in public understanding of science and the "linear model" in science-policy interaction. I think these are something similar. The former is that people will behave more wisely if they have got more scientific knowledge. The latter is that better policy will be formed whenever science yields more novel pieces of knowledge.

I think that most contemporary social scientists hold both of them as already debunked. They tend to dismiss scientists and science policy makers who just push scientific knowledge alleging that they follow those falsified (=proved to be false) theories.

Perhaps we should replace them with more interactive models, especially when the science has inherent uncertainty. But I do not think that a model which cover all cases have been completed.

I suspect that these old models are sometimes successful. Otherwise they would not be popular once upon a time. Perhaps we should rescue these models with explicit notice of the range of their applicability.

I am no expert of social sciance and I am not confident about my story here.

ac said...

John F Pittman,
People still seriously call themselves 'frequentists'? How do I apply to join the Frequentist Society?