The Honestly Broken

In our last installment, I pointed out that there's a clear gap in Roger Pielke Jr.'s taxonomy in The Honest Broker: there is no role for anyone trying to remove bad science from the conversation. If one's alternatives are a) to advocate for a particular policy or b) to maximize available choices among policies there is no expertise-informed winnowing; bad policies which sound good cannot be sifted out.

Here I will look more closely at the central idea of The Honest Broker, and will argue that it, itself, constitutes invalid scholarship, and it, itself, should therefore not be given great weight in the public discourse.

Obtaining the Taxonomy:

Roger begins by proposing two models of science (linear vs stakeholder model), and two of democracy (Madisonian vs Schattsschneiderian). I won't make an effort to repeat his definitions; I find them hard to understand and so I would mess it up. I will simply note that each possible quadrant of "model of science" + "model of democracy" yields a role:

linear/Madisonian: Pure Scientist (does not care about policy)
stakeholder/Madisonian: Issue Advocate (evaluates science to support an agenda)
linear/Schattschneiderian: Science Arbiter (responds to questions from policy sector)
stakeholder/Schattschneiderian: Honest Broker (clarifies and expands choices)

Re-Obtaining the Taxonomy

Roger obtains a separate taxonomy by asking a question contingent on the answer to another question:

Q1: Is the decision context characterized by values consensus and low uncertainty?

If yes to Q1:

Q2a: Is the decision connected to policy?

If yes: Science arbiter

If no: Pure science

If no to Q1:

Q2b: Reduce scope of choice?

If yes: Issue advocate

If no: Honest Broker

Roger's Unproven Theorem

The two taxonomic systems yield the same results via very different means! The first taxonomy is based on your model of science and your model of democracy; it produces exactly one role for each possibility. The second taxonomy is based on your model of the specific question at hand, and then conditionally (in the Q2b case) on the motivation of the scientist or (in the Q2a case) on a further elaboration of the model.

To make matters even more confusing, one of the combinations seems so trivial as to be unnecessary: if a question has values consensus and low uncertainty, and only then, this taxonomy yields a "Science Arbiter" (*), which apparently is a very easy role. A Science Arbiter is apparently someone who tells people to do what they have already decided what to do. Is that a paid gig? Where do I apply?

(*) Fixed - see comments

An exasperating part of all is the "reduce scope of choice" question. This is a useless question to an informed person with an interest in policy. The informed person is interested in finding the best policy consistent with his or her information (increasing scope) and in avoiding policy that is not consistent with his or her information. The idea that one must choose one role or the other is not helpful. It is exactly this lack of a meaningful role for someone who is neither advocate nor neutral but actually engaged and informed that is a core problem I often complain about; and yet Roger eliminates such a role by construction.

But consider what has just happened: the same four-way taxonomy was achieved by pursing two very different paths. For instance, a person who works on a problem with either high uncertainty or high values contention who wants to increase the available scope of responses is someone who has a stakeholder view of science and a Schattschneider view of democracy. Your model of democracy and your choice of scientific pursuit are tightly coupled with a perfect constraint!

Now, you would think that getting to the same place by two different means would itself be material for a book. What an extraordinary coincidence! A deep and unintuitive theorem must be involved!

But no, the equivalence is merely stated without proof or even any expression of surprise or delight. This is presented in the second chapter and never defended. It was clear to me at that point that anything a scientist might take for clarity of thought was lacking in this book. After that I read more quickly but with less energy and enthusiasm. This kind of handwaving does not reward a careful reading.

House Built on a Weak Foundation

The rest of the book amounts to working out the implications of this taxonomy. But the taxonomy is confused. In particular, no realistic example of an "honest broker" is provided, and no really compelling definition is offerred either. We are only told that honest brokers are needed, that their role is to maximize the number of options available, and that in addition to advocates they are all that is needed.

The many glowing reviews Roger has received for this book, in my opinion, do not speak well for the state of discourse among the community in which he participates. Of course, it's well known that the sociology of science has a history of entertaining the most absurd relativism; compared to Bruno Latour, Roger is another Bertrand Russell. But, alas, compared to Bertrand Russell, he is another Bruno Latour, too.

Again, this is not to deny that Roger has identified a genuine and crucial issue, which I believe he has. Also, I think he has made some interesting observations along the way. But unfortunately, he has not adopted a scientific attitude of self-doubt. His ideas appear to be merely stated, and nowhere tested. They are also stated rather vaguely, so the the implausibility of reaching the same taxonomy from two completely different sets of dichotomies is not especially striking, unless you actually pause to think about it.