It is time to stop quivering in our boots in pointless fear of the future and just roll up our sleeves and build it.
- Ray Pierrehumbert

Monday, December 14, 2009

Data access & collaboration in weather/climate

First session at AGU:

Next generation distributed data access diagnostics and collaborations for climate and weather models

Sort of an ironic session, with everyone presenting the need for srandardization in the field but everyone proposing different and mutually incompatible standards. Everyone ignored ESMF, interesting in itself. By far the nicest and most convincing presentation from where I sat was Reagan Moore's irods (irods.diceresearch.org).

The irony: the collaboration groups all ignore each other! There's an essay here about how science has learned the wrong lesson from commerce. Competition itself is neither necessary nor sufficient...

4 comments:

Jim Bouldin said...

Depressing.

I think you touch on a VERY big issue in science in general: lack of a comprehensive, goal oriented plan at solving problems. Although one could argue that independent research groups coming up with broadly coherent answers to the same question is a positive thing (and it is), so much time and effort is wasted in the process, either by redundancy, or reinventing the wheel, or going off on tangents under the lure of publishing something "novel".[great, novel but useless!]

This same problem affects the literature. Far too much emphasis by different groups in generating some result, any result, than in first working to define, develop, categorize, and improve analytical methods. Same for data collection efforts. Work these things out, build the foundation first, then apply the methods and data to address certain, defined problems. Too much cart before the horse, and competitive ego crap, and it drives me nuts. More nuts that is.

David B. Benson said...

This happened already in the IT industry. Eventually people converged on a single standard when the matter became more urgent.

When one starts standardizing, it may not be cler what the "best" standard is. Consider the QUERY keyboard, for example...

Oale said...

so, if we know something isn't solvable in reasonable time (f.e the Navier-Stokes eqs), we should either stop studying fluid dynamics until someone solves them(f.e. by attaching a version of Kolmogorov microscales to QM?), and stop trying to find various ways (i.e. duplicating the effort) to approximate these (minimizing Kolmogorov complexity of the projection)? This sounds like requiring the fluid dynamics to work like a clock. Well, a clock will eventually break too (the time of which is one more thing to be estimated). There is no need to invoke entropy (IMHO, the 19th century way of saying:"mechanics doesn't always work") in any goal oriented system, since the laws of thermodynamics state no work can be aquired from energy assigned to entropic effects in this world. So in fact, if one wants to get some work done in climate science, one should try to minimize the use of entropy... Looks like this is evolving to the argument:"The models are the best we have", so I'll stop.

Michael Tobis said...

The sun was shining on the sea,
Shining with all his might:
He did his very best to make
The billows smooth and bright--
And this was odd, because it was
The middle of the night.

The moon was shining sulkily,
Because she thought the sun
Had got no business to be there
After the day was done--
"It's very rude of him," she said,
"To come and spoil the fun!"

The sea was wet as wet could be,
The sands were dry as dry.
You could not see a cloud, because
No cloud was in the sky:
No birds were flying overhead--
There were no birds to fly.

The Walrus and the Carpenter
Were walking close at hand;
They wept like anything to see
Such quantities of sand:
"If this were only cleared away,"
They said, "it would be grand!"

"If seven maids with seven mops
Swept it for half a year.
Do you suppose," the Walrus said,
"That they could get it clear?"
"I doubt it," said the Carpenter,
And shed a bitter tear.