An Interesting Gripe

Never one to shy away from sticking his neck out, Ray Pierrehumbert (aka Raypierre) has actually submitted a top level article to Dot Earth. Ray's follow-up comment is especially interesting.

There is so much uninformed comment here I don't know where to begin. So many people are complaining about the "open-ness" of climate modelling, when in fact it is just about the most open area of science there is -- certainly more open than the economic modeling used to make trillion dollar decisions routinely. People complain about data sets not being released, when the data in question represents only a tiny slice of the total. You can get virtually all of the data from public sources. And do you want to see what's in a climate model? The algorithms used in the models are all documented in the peer reviewed literature. For most of the key models, you can get the source code and technical documentation and look at it for yourself -- many of the models even will run to some extent on a laptop.

So before you go on declaring a fatwa against climate scientists, why not take some time to get to know us? What we are like, and the way we do our work, bears no resemblance to the hateful cartoon you are pitching. And what do you think any of us would have to gain from a conspiracy to distort climate data anyway? If we were interested in making money, we could do it a lot more easily by just becoming investment bankers. Most of us are in this because we think the world is facing real peril, and we want to understand the nature of that peril better. A better world for my grandchildren to live in. Your grandchildren, too. Everybody's grandchildren, everywhere. Is that something to hate us for?

Now it's especially interesting to me because I do know Ray. I have confidence in his moral integrity and tremendous admiration for his intellectual capacity and domain knowledge. The trouble with this "getting to know Ray" (or someone like Ray) algorithm that Ray proposes is that, as any computational person worth his salt will immediately point out, it doesn't scale worth a damn. It is an effective technique at small scale, though, and I heartily recommend it to those few people who get the chance.

But it's also interesting what response this gets. "Biker Trash" replies

" There is so much uninformed comment here I don't know where to begin. "

Well, let's take a couple of specific examples that illustrate why this might be the case.

" So many people are complaining about the "open-ness" of climate modelling, when in fact it is just about the most open area of science there is -- certainly more open than the economic modeling used to make trillion dollar decisions routinely. "

Let's compare the open-ness of climate science with the degree of open-ness of the independent review and verification of, the Yucca Mountain Project, the certification of passenger aircraft by the FAA, construction of a nuclear power plant by the NRC, approval of a new drug by the FDA, construction of bridges, elevators, buildings, and many other cases that effect the health and safety of the public. This is the standard for the degree of open-ness required for all decisions that have the potential to effect the health and safety of the public.

Consider the Yucca Mountain Project. Every calculation, every piece of data, every detailed aspect of the independent review and verification of this project is online and accessible to anyone. Every nitty-gritty detailed part of every aspect without exception; http://ymp.gov/. Note that the basis of the Yucca Mountain Project is a natural process, directly analogous to that of climate science.

The same degree of open-ness is true for construction of nuclear power plants; check the NRC Web site; http://www.nrc.gov/reactors.html. The complete independent review and verification, not the extremely limited and incomplete information that appears in peer-reviewed literature, is publicly available.

Where is the documentation that the procedures and processes used to ensure that independent review and verification of climate science results have been properly applied. Where is the documentation of the procedures and processes that are required to be applied.

" And do you want to see what's in a climate model? The algorithms used in the models are all documented in the peer reviewed literature. "

The phrase, "the peer reviewed literature" is not very specific and the climate science literature is enormous. We have been told several times the same thing that you say here. Several people have tried to find the following information without success. Because climate science is your field of expertise and experience you could save us hours and hours of additional time if you could give at least a hint or two having more specificity.

BT goes on a bit of a tangent about computational fluid dynamics; an understandable and informed error. Many people think of climate physics as primarily a classical problem in fluid dynamics; they are misinformed in some fundamental ways that few people are qualified to explain in a way that will be acceptable to those within the field and convincing to those outside it. I have tried with very limited success and I won't try again just now. (In short, getting the fluid dynamics right to high order is not among the key problems. We aren't trying to solve a well-specified system. We are trying to specify the system.)

But the part of the complaint above, as far as I am concerned, is perfectly valid. Given the importance of climate modeling, one would expect a more rigorous and contemporary development process. Steve Easterbrook (along with his student Jon Pipitone) is the first person I know from the software engineering community to really take the development process of these models at face value and understand the some of the constraints that make the codes work the way they do. But Steve is just scratching the surface. There are efforts from within the community to improve matters, but I am among those who find them misguided.

As matters stand, it requires weeks to understand how to effectively run a climate model on a supported machine, and typically months to get it working on a new one. The success rate closely depends on how well integrated into the community one finds oneself. An outsider downloading a few blocks of code with no guidance will get nowhere.

So Ray's claim of openness reminds me of the part in the Hitchhiker's Guide where the bureaucrat defensively stated that Arthur's public condemnation had been duly publicized when the notice was in a locked cabinet in an unlit basement in a disused lavatory with a sign on the door saying "beware of the tiger!"

The thing is, climate scientists don't expect better, and never have seen much better. The fact is that despite the endlessly inflating claims of billions of dollars going to climate science, the software engineering staff on the leading American climate model has been reduced from eight to six, with the cuts coming from user support and documentation. This is why CCSM, though on release 3.1, is only documented to 3.0, and that only badly.

Yes, it's true, it's a trillion dollar problem, but the annual budget for studying it (once you subtract all the impact studies and the ecological observation programs and the satellite launches and satellite base stations, all very valuable but not what most people think of as climate science, and get right down to what is actually climate science) is about that of a high-profile Hollywood movie, and that for model development and maintenance on each individual model is far less. From 2003-2008 NCAR had to lay off approximately 55 people and lost another 77 positions due to attrition, totaling roughly 16% of NCAR positions, because of sub-inflationary NSF funding and decreases in other agency support. (Statement of program reductions at NCAR).

Here's a presentation of how NCAR is meeting a 9 million dollar shortfall this year. (PDF)

Yes, the support for the software used in climate science sucks. I promise you I hate it more than you do. Maybe if you all would stop treating climate science as a heinous enemy and supported improvements, or hell, even complete rewrites, things would get better.

What you see is what we get every bit as much as it is what you get. The climate community should stop pretending to any practical openness; in fact the only way to play is to sign up for a tour of duty. This is not because we are hiding anything. It is because we lack the resources (and to some extent the skills) to do any better. What is perceived as an intent to hide is mostly an incapacity to elucidate, exacerbated by some excesses of competitive zeal which come from a tightly competitive grants process.