"Our greatest responsibility is to be good ancestors."

-Jonas Salk

Wednesday, October 10, 2007

Fallibility of Consensus

I think it's ludicrous to claim that scientific consensus constitutes scientific fraud, as Crichton alleges and as many people have echoed. In fact, I believe that consensus is how science operates; ultimately it is a flawed social process that tends to reveal truth.

The circumstances under which the consensus process succeeds or fails are very important, and are not at all easy to investigate.

The social momentum of a given academic community can deflect progress. I had hoped this would be increasingly rare, but perhaps it isn't. It may depend to a large extent on the health of the scientific culture.

I had a brush with a minor example of this myself. It happens that my thesis advisor, John R. Anderson (tellingly, now at Pixar) recommended a topic for me where I would disprove the common wisdom about a relatively obscure atmospheric phenomenon, called the Madden-Julian oscillation. I never quite understood what John was going on about, to be honest, so it was problematic right there, though he convinced me that the prevailing wisdom had to be wrong. However, nobody except those who have already published cares enough about the MJO to care very much about an alternative hypothesis. So this would be a bad career move, even if John is right (as he tends to be about such matters). It would make a few enemies and no friends. I moved on. John's hypothesis remains published only as grey literature from Colorado State. Does it matter? Mostly as an instance; the consensus is sticky on matters of limited importance.

The New York Times has an interesting article by John Tierney, subtitled "A Severe Case of Mistaken Consensus" on this matter, claiming that the relevance of a high-fat diet to heart disease, long held as a consensus in the US, is uncertain. I am having some difficulty judging the quality of the article or of the truth of the matter at hand.

In the discussion area, Tierney's readers are, unsurprisingly, quick to bring up "Al Gore" (the new word for "Global Warming" apparently). Unsurprisingly, given the vagueness of Teirney's reporting, some or unconvinced of the point being made.

To be fair to Tierney, the point is Gary Taubes' (no relation) point, which he is making in the book Good Calories, Bad Calories, which Tierney is reviewing.

On the other hand, one commenter mentions Tierney in the same breath as Easterbrook, which is perhaps unfair guilt by association, but worth considering:
Have Tierney, Gregg Easterbrook, and John Stossel ever been seen in the same place as the same time?

They all seem to pay their bills by claiming that they know more about science than the overwhelming majority of scientists. They are journalists, and not very good journalists, at that.

Outrageous, undefendable strawmen and the appeal to an oddjob, outspoken authority are their stock in trade.
Regardless, the article does discuss the dynamics of incorrect consensus very effectively and is worth a read on that account.

Please don't misunderstand. Consensus itself is not a sign of science misfiring. I am acknowledging that consensus is not a sign of science working correctly either. It's more subtle than that, unfortunately.

And regarding Taubes' point, it's certainly argued that he suffers from the same selectivity of evidence that other consensus-busters are prone to.

As for myself, I was influenced by Taubes' article in the Times a few years ago to take up very low carbohydrate dieting. I lost some weight episodically but in the end it was just an oscillation. And a few weeks ago I had a kidney stone episode. The doctors were puzzled because they claimed I was rather older than a typical first onset of this problem. It's associated with excessive animal protein. So I am regrouping. Count me as one anecdotal argument against Atkins, though.

All the same, the arguments against consensus have some cogency.

Economists have a number of consensus opinions, including that theirs is the only science that is qualified to synthesize all other sciences for the policy sector. The fact that some belief is a consensus in some academic community doesn't make it true.

Let me make sure this is clear.
  1. I think the IPCC WG1 report does represent a consensus position among the relevant sciences.
  2. I think the IPCC WG1 report is a good representation of truth.
  3. IPCC report represents truth not because there is consensus. Consensus exists because of the same intellectual and social robustness of the relevant disciplines that allows them to approach truth.
  4. The extent of the robustness of a scientific community is difficult for outsiders to determine.
  5. Yet that determination is crucial to successful governance.
Meanhile I'm still fat and we're still building coal plants...

7 comments:

Rob said...

I've been wondering: who was the first group to use "consensus" in the global warming fake-debate? Did scientists use it to defend their position or did deniers use it to say "the consensus is wrong".

I'm starting to think that arguments over weather or not there is a consensus, and "counting scientists" in various surveys, is a side-show, a way to avoid dealing with facts. It goes something like this: the science is too impossibly hard to understand so lets try to establish what a majority of scientists think and go with that. I don't think scientists should participate in this side show. Instead, work to make the facts more understandable since they tell an unambiguous story.

Michael Tobis said...

The IPCC process is a formal consensus process and so there has been an effort at a formal consnesus ongoing for almost twenty years now.

Working to make the facts more understandable is explicitly the goal of this process.

Rob said...

Let me be more clear. I wasn't asking for a history of the conensus process. Rather I'm asking for the history of pointing to a consensus, not necessarily the IPCC report, to try and end a GW debate. Because pointing to the IPCC report and its hundreds of authors leads to "counter consensus" b.s. like the Oregon Petition.

Anonymous said...

I think Rob gets it exactly right. My shortcomings in science history reserved, I never heard the law of gravity or law of special relativity being defended by "it is now the consensus of the scientific community". Or any other hypothesis in hard science.

If you ("you" as in climate science community) want to make it harder to be a denier, or convince healthy skeptics like me, you should make more effort to communicate your science on many levels of scientific understanding, and be stringent about reproducability. (Personally, I/d be thrilled to read books you recommended in an earlier post about climate modelling, but for a starving graduate student in a different field, it's not an expense I can take at this moment.)

Michael Tobis said...

Frederik, I would like nothing better than to be able to complu with your request. However, there is no way to be paid for the service you request.

Meanwhile, there is employment for people who are skilled at one-sided argument and obfuscation.

This is why the outside view of the matter is confused. It is not primarily the fault of the practitioners.

Anonymous said...

I happen to have been studying this of late, so let me try, including useful Web references to avoid books.

1) A general process and chronology covers a number of cases (not all), so I'll do that first, and then answer the specific question about consensus in global warming.

a) First, nobody really knows.
b) People start doing serious studies, and they don't all agree, but evidence starts to pile up around some hypothesis on its way to being a theory.
c) As elements of a hypothesis get established, people stop arguing about them, but argue about other elements.
d) At some point, some things are so well-accepted that almost no one doing serious research and doing peer-reviewed publication argues with it. I say almost no one, because there are always a few dissenters, sometimes even quite distinguished ones.

But at that point, the consensus is as strong as it ever gets in science, and of course, by then it's wired into textbooks with few if any caveats. Researchers don't normally run around proclaiming consensus all over the place, because by the time it is a consensus, it's obvious, so who cares? Only if new data appears that truly contradicts the consensus does anyone get excited - of course, overturning a consensus with a new hypothesis that gets confirmed ... is a giant win for a scientist.

Well, that applies to most scientific arguments ... EXCEPT:

2) Sometimes people find the resulting consensus undesirable for extra-science reasons:
a) religious
b) economic
c) ideological/political
or combinations.

and in this case, a common strategy is to say: "there's still a lot of argument" or "teach the controversy", and these have tended to use the small number of scientists who dissent plus a lot of PR, not to change the science, but to create doubt in the public.

For example:
- Evolution vs creationism/intelligent design, going on for a long time. (a)

- Medical science versus smoking and then second round, secondhand smoke. (b) If you can find Allan M. Brandt's "The Cigarette Century" you can learn all about the tactics used to obfuscate the science, and it's the best case, because unlike almost any other case, there is this huge database of public records of internal documents.

PR agencies honed tactics including use of the few dissenting scientists, creation of captive scientific-sounding front organizations (like TIRC, or later, TASSC - The Advancement of Sound Science Coalition), creating doubt, etc.
(As an example of a dissenter, the great statistician Sir Ronald Fisher disbelieved the smoking-disease linkage right up to his death in 1962, long after there was very powerful evidence.)

A bunch of thinktanks came into existence, learned the skills with tobacco, and then applied them later to:

- ozone depletion vs CFCs (b,(c))
- other environmental regulations vs business interests (b, (c))

and of course,
- global warming vs fossil fuel (b, (c)

The (b, (c)) means that the primary impetus and funding tends to be (b), but there is often an ideological connection (c), which for these cases tends to have certain sorts of current-conservative and/or extreme libertarian leanings. I.e., this is not isomorphic with the Republican Party, but tends to be closely tied there at the moment.
Some of this dates to Frank Luntz, an influential pollster: see Wikipedia entry and the parts of LuntzSpeak, especially:
http://www.luntzspeak.com/memo4.html

Hence there is: George C. Marshall Institute, SEPP, CEI, AEI, Heartland, Frontiers of Freedom, SPPI, Fraser Institute (Canada), (and more) mostly in USA, and concentrated around Washington, DC, and with a lot of overlapping participants, often with funding from tobacco companies, very-conservative foundations, ExxonMobil, and coal companies.

3) In any of these, we get into a state where the scientists think there is a strong consensus about something, and will say so if asked, but are working on areas that are in doubt. Papers do not waste words reaffirming the consensus, i.e., very few biologists waste words confirming evolution.

However, the other side constantly attacks the consensus, trying to sow doubt among the public, "teach the controversy" etc. Fighting with this is generally a thankless task for most scientists, as Michael says.

Note that this very different from a real scientific argument, like the multiple-decades-long fight over continental drift, or the shorter fight over the causes of ulcers.

When somebody says there is a consensus, it's not usually that they're trying to close off scientific debate or win an argument, it's just an observation of fact, as seen in the rear-view mirror! Scientists I've talked with (a lot, over 40 years, on many different topics) usually calibrate what they say with uncertainty levels.

Sometimes, the obfuscation strategy backfires into a court case like "Kitzmiller v Dover Area School District", which is one the Discovery Institute did not want to fight, and in which the dissenter witnesses, under oath, got slaughtered in court in front of a conservative, but honest judge John Jones III.

4) AGW consensus.
Assuming this means: "The recent (say since 1975) rise in temperatures is mostly caused by humans, especially by adding GHGs to the atmosphere." I think that has actually been mostly in place (scientifically) before 1990, although as always, people argue about the details.

REPUBLICAN President George H. W. Bush said, in:
http://www.presidency.ucsb.edu/ws/index.php?pid=17765
"President Bush announced today that the United States has agreed with other industrialized nations that stabilization of carbon dioxide (CO2) emissions should be achieved as soon as possible." that was 1989.

The more it looked like people might actually do something to conserve fossil-fuels, and the stronger the statements got by the IPCC, the louder the anti-AGW PR got, and for whatever reason, some parts of the Republican party took a distinct anti-science turn (see Chris Mooney's book "The Republican War on Science."

5) At the February 2004 AAAS meeting, the (prestigious) George Sarton Award lecture was "The Scientific Consensus on Climate Change: How Do We Know We're Not Wrong?" of which you can find a slightly-updated version at:
www.ametsoc.org/atmospolicy/Presentations/Oreskes%20Presentation%20for%20Web.pdf

This was given by UCSD Professor Naomi Oreskes, who is a well-published, award-winning geoscientist and science historian, and it is well worth reading.

Naomi's original talk mentioned a quick experiment she had done, which was searching the ISI Web of Science for "global climate change" in papers published 1993-2003 and looking at abstracts and assessing them, expecting to see when and how the consensus got established, and was surprised to find that the consensus was already there.

However, this got a strong audience response, and she ended up doing a 1-page essay for Dec 2004 Science (which does not give out pages casually):

http://www.sciencemag.org/cgi/content/full/306/5702/1686

Bennie Peiser (a UK Social anthropologist!) wrote a letter to Science (not accepted) attacking these results, in 2005. It turned out that he simply lacked the expertise, and this was refuted, rather thoroughly.
http://timlambert.org/2005/05/peiser/

The other side constantly raises doubts about the consensus, and often attacks Oreskes, sometimes quite personally via threatening letters.

The latest one is a bizarre combination of a Britsh Lord Christopher Monckton, 3rd Viscount of Brenchley, a London endocrinologist, Klaus-Martin Schulte, who wrote an article (not yet published) that tried to refute Oreskes and teh consensus again. Even without being published, it managed to generate:

Google: less than half published scientists endorse global warming
--> 700,000 hits

I.e., regardless of the truth, the desired publicity was accomplished, using a well-tuned PR machine centered in Washington, DC.

There also turned out to be plagiarism, threats of lawsuits, letters sent to Oreskes' Chancellor and then publicized via Business Wire, etc, etc.

So: there is a large machine crying "no consensus, there is doubt" which is perfectly happy to publicize
- an indirect reference to a paper by an endocrinologist
- who turns out to be fairly clueless about climate science,
- and whose paper couldn't even get published in the poorly-regarded journal to which it had been submitted.

This story is spread across various blogs, but some key ones are:
http://scienceblogs.com/strangerfruit/2007/08/oreskes_responds_to_schulte.php
http://scienceblogs.com/strangerfruit/2007/09/ah_another_day_in_denialism_la.php
http://scienceblogs.com/deltoid/2007/09/oreskes_replies_to_schulte_1.php
http://scienceblogs.com/deltoid/2007/09/schulte_replies_to_oreskes.php

Anyway, that wasn't a simple answer, but I think captures the history.

bigcitylib said...

Frederick wrote:

"I think Rob gets it exactly right. My shortcomings in science history reserved, I never heard the law of gravity or law of special relativity being defended by "it is now the consensus of the scientific community". Or any other hypothesis in hard science."

Depends what you call hard science. It is the very strong consensus of the vert. paleontological community that birds are teh descendants of dinosaurs. The "C" term appears in both the popular and technical literature on the subject, as do "appeals" to the consensus.