Consistency

You may remember some confusion betwixt James Annan and Roger Pielke Jr regarding "consistency", here, among other places.

I tried to get Roger out of the mess he was in thus (linked article, my comment dated May 20, 2008 09:57 AM):

Roger: "But I will repeat, any test of consistency that means that more bad forecasts (ie. a greater spread in your ensemble) implies greater consistency is just nonsense. It means that the problem has been framed in a way that is at best misleading."

Technical meaning may not always match intuitive meaning, but we are scientists and should not resort to the sort of argument where a "tightly constrained" result is one that requires liberation.

It is indeed the case that every possible result is consistent with a theory that anything can happen, and that very few results are consistent with a very tightly specified theory. This doesn't mean that vagueness is *better* than precision; it just means it's *harder to falsify*. Frankly, one doesn't expect a scientist to construe this as "misleading". On the contrary, it seems rather obvious.

Essentially, it's easier to satisfy a vague promise than a precise contract. The question of consistency is raised by you and others questioning the modeling enterprise, not by the modelers. The idea seems to be that observations call the model projections into question at the global scale. To make that claim you have to show that the observations are inconsistent with model predictions. You can't actually get very far doing that because the GCMs make very modest claims about near-term global temperature change.

If there is any misframing you are the one who set out the frame in the first place.

to which he immediately made the surprising response:

If you are suggesting that what these scientists really mean when they say this is that "models make very weak claims," then I'm surprised, because when I see such statements I interpret them to say quite the opposite.

Given the technical context of his discussions up to that point, this seemed strikingly naive. Does RPJr represent the scientific community or the lay public in these discussions? But his closing point in that comment is worth considering.

Such claims by scientists and the IPCC lead people in the professions of policy research and decision making to ask, "what could these scientists mean with such claims?"

When efforts to resolve this question are met with responses that "technical meaning may not always match intuitive meaning" then we have a problem. If the IPCC is indeed to support decision making then it should present its findings in ways that make intuitive sense to decision makers. Consider this exercise an effort in that direction.

Obviously, we are not there yet.

I didn't pick up that ball, partly because I was still shaking my head about the technical discussion. But there is a real point to be made here, and RP Jr takes up the opportunity to do so, in going after a pair of apparently contradictory papers. (To be honest, after a pair of press reports. Here are the actual papers, with a h/t to ThingsBreak on Planet3.0)

http://www.pnas.org/content/early/2010/02/09/0915062107.abstract
http://ams.allenpress.com/perlserv/?request=get-abstract&doi=10.1175%2F2008JCLI2111.1

It's hard to see those results as both being valid.

In this case, these could just be two perfectly reasonable studies reaching contrary results, but one predicts a warmer coast and lower condensation level, while the other predicts the opposite. So unless my reading is wrong, at least one of them will turn out to be wrong. There's nothing at all unusual there. Regional climate change is hard, and the occasional disagreement of this sort is not totally unexpected. Though some easy chuckles might result, it's nothing for the world (aside from the respective authors) to lose much sleep over.

But there's the larger point about "consistency".

Though he messed it up terribly in his argument with James, RP Jr. has a real point in the whole "consistent with" language, which is used in two very different senses. Sometimes it is used to mean "provides supporting evidence" (see WG 2 press releases) and sometimes it us
used to mean "not inconsistent with" i.e., "does not provide sufficient evidence to refute".

I really would like the "consistent with" language to disappear form public communication; at this point it has become terribly unclear what it means.

The AR4 WG II SPM says " Of the more than 29,000 observational data series,[7] from 75 studies, that show significant change in many physical and biological systems, more than 89% are consistent with the direction of change expected as a response to warming (Figure SPM.1)
[1.4]. " which is clear enough, but their press release said something like "more than 89 per cent of the significant changes in physical and biological systems are consistent with global warming." I couldn't dig it up, but here's Susan Solomon saying

"89% of current changes in ecosystems are consistent with changes
expected due to global climate change"

http://www.iittl.unt.edu/IITTL/itest/msosw_web/Global_Climate_Change.ppt

Now if 11% of your observations are NOT consistent with your theory, you have a big problem. The trouble is, the implicit contextual meaning of "consistent" is lost here, and so the claim is not helpful.

I don't think having two contradictory regional studies is all that embarassing, especially on a regional scale in a complex climate, but I agree that they are contradictory. I also think, regardless of literal truth, "consistent with" like "significant" or "global warming" itself, is a phrase we should be wary of in communicating to the public.

This isn't hair-splitting. This is a real point of weakness in public communication and it needs to stop.

The question of when to take RPJr. seriously and when not to remains, to my eye, complicated. I agree with Joe Romm that a press article should not quote him as representing "climate scientists". On the other hand, he is a close observer of the scene and his critiques of the communication skills of climate science have merit.

Roger Pielke Jr. is very far from relaibly insightful, in my opinion, (for that matter, neither am I) but I think it is a mistake to demonize him. In this case we ought to thank him for continuing to press a very valid point despite having gotten into a tangle on it in the past.

Note: I have already violated my "ignore Roger Pielke Jr." policy by trying to track down the "hot spot" critique and getting caught up in the morass of Klotzbach foolishness. (I stand by my earlier conclusion that the oft-cited Klotzbach et al. "bias" paper, coauthored by Pielkes Sr. and Jr., is a mess and that the naysayer spin on it is totally valueless.) In fact, as long as the relationship between climate science and the press is on my beat (though I am no competition for some of the amazing metamuckraking being done at Deltoid these days) and so long as the press remains fascinated by RP Jr., avoiding him entirely verges on impossible.