It is time to stop quivering in our boots in pointless fear of the future and just roll up our sleeves and build it.
- Ray Pierrehumbert

Wednesday, May 26, 2010

Go Read This, OK?

This one's for Andy.

I don't know Al Giordano from Adam, and I don't know why he calls his site the "Narcosphere" and I don't think I care. But he gets this so exactly right I'd like to quote the whole thing. That wouldn't be fair, so here's a good quote:
Without an easy solution in sight, and with the knowledge sinking in of just how harmful this oil gusher will be to the Gulf of Mexico, its shores, its fishing and tourism and quality of life, a lot of people seem to be screaming that somebody should yell louder and point their fingers harder.

Okay, just this once, I will point fingers. You know who is to blame in addition to BP and the government that allowed this oil rig to be built?
Go follow this link to find the answer to that question, and lots more about how you can help.

Update: My gripe is with people demanding that the mess be undone by Obama. As a commenter on Giordano's site said, "I worked to elect a president, not a glorious magic dictator."

Calling this Obama's Katrina is ridiculous. Nobody expected Bush to reverse the hurricane. They just wanted water, food, medical support on the scene. In this case, everything feasible is being done after the disaster.

It is true that in both cases the disaster was caused by laziness in government administration tracing back to misallocation of resources by government policy. In both cases it was neglect of consequences of foreseeable events.

Update: Brad Johnson has a much sounder case here, when he argues that everything except capping the well should be taken over by the government, regardless of whether the capping operation works. There is a case to be made that BP's problems aren't just a matter of bad luck, but of bad process, and that it should therefore be ineligible for government contracts.

Brad's focus on the "foreignness" of BP rubs me the wrong way, though. It's not as if Americans expect or want US companies to be treated badly overseas, is it? If Exxon/Mobil had this record, should we treat them differently?

It's clear that the incentive structure wasn't sufficient to seriously get a safety-minded culture at BP, or at least at BP America. How best to handle that is interesting.

My view is that large corporations, which should not be considered "persons" for purpose of rights, should for comparable reasons not be considered "persons" for purposes of ethical responsibilities.

That is not to say that BP isn't liable for damages or penalties. It is to say it is meaningless to get mad at BP; it is not a morally responsible entity. The incentive structure has to somehow go beyond the corporation and to the people who run it.

How? Social pressures and transparency where possible, and well-thought-out regulations and incentives.

It would not be impossible for a concerted effort to succeed in destroying or greatly diminishing BP as an organized entity over this. Would it be a good idea? I don't know. I think it would depend a lot on how and why this was done.


jg said...

I don't understand your identification with AG's post. Granted, we all drive a demand for energy, and solutions rather than yelling and blame are needed. But his argument seems to be comparable to blaming us all for an outbeak of salmonella because we drive the demand for food. Rather, we empower government to oversee health and safety and to manage risks like drilling in deep water. This oversight seems to have broken down in the deepwater horizon disaster with no tested contingency plan for this worst case. What's wrong with instilling a little fear in politicians over an accident of this size? Perhaps I'm not getting his point.

Michael Tobis said...

We're discussing how to handle the actual, real, physical problem. The way to do that is to get the best people on it and give them lots of resources. As far as I can tell that is already happening.

Yet people seem to be calling for a shakeup and a new way of handling the emergency that does not actually involve petroleum engineers. That is, certainly not going to help.

There's no telling what the next couple of days will bring. I have no doubt that there were massive screwups before this event, on the part of BP and/or the subcontractors and pretty much for sure the regulators.

I have little doubt that pretty much the best possible effort is happening now, both in remediation and in repair.

Others disagree. Fine. Let them propose something besides waving their hands and pleading for something "better". What, exactly?

jg said...

thanks for clarifying and for the updates.

John Mashey said...

Part 2 (from earlier thread).
"Real solutions might look towards nuclear submarines..."

A. Examples.
Nuclear submarines : Rickover was a perfection fanatic. Computers work because systems designers are fanatic about considering fault paths and dealing with them.

Sales compensation plans try to get people to do the "right thing", which includes avoiding long-term negatives. Good executive compensation schemes don't just pay people for the current quarter, but have longer-term components [a rule often violated by Wall Street financial places.]

Extractive industries often favor a "Take the money and leave" approach. Other behavior can be found in industries that have to think longer-term, *cannot* leave, and must seriously weigh first-cost versus maintenance costs. With all its plusses and minuses, the old Bell System certainly worked that way. It had specific metrics to make sure local repair managers didn't jazz up their results for a given month saving money by slacking off on preventative maintenance, because we knew that would hurt in the long-run.

Design of switch software and hardware was fanatically dedicated to reliability, with crazed targets like "no more than 2 hours downtime in 40 years.".

In 1975, a big switching center caught fire in NYC. What was done to fix it? There was some redundancy, there were contingency plans. Specialists were mobilized from all over the US. Switches were diverted from planned installations. It is hard to see how anyone else could have helped much.

Bell had long had operations centers that set up to recognize and manage problems. I once visited a special one buried in a NJ hill. When you took the elevator down, you entered a room that like the "War Room" in Dr. Strangelove. It had switches on springs, designed to survive shock from nearby nuclear blasts.
Monopoly money helps of course, but so does long-term thinking.

B. Solutions.
1) It *is* possible to have large organizations be very effective in long-term thinking about safety and cost-tradeoffs.

2) But it must be part of the organization from top to bottom, and the incentives have to work at *every* level. In a startup, it is possible to put together a team of nothing but A or even A+ players, but that is simply impossible in big organizations - one must enable and incent large numbers of B&C players to do the right things, routinely. This is not rocket science - after all, the armed forces ahve a long history of doing this.

3) If there is a temptation to save money by cutting corners on safety issues, in the hopes that you'll get away with it, or that if there are problems, they show up after you've moved on ...
somebody has to prevent that either by management tracking or regulatory enforcement. It has to be *very quick* feedback, not the chance of blowup that may happen years later. Bad safety practices need to hit *this quarter*'s bottom line, and a lot of people's quarterly bonuses, and the hit has to be noticeable. The hit can't be deferred N years by legal wrangling.

4) Insurance companies are in the business of pricing risk. If a company *really* has to pay the costs for cleaning up an oil spill, *and* the true costs of economic damage, *and* the true costs of ecosystem damage [in increasing order of difficulty], maybe they should be required to carry insurance, as states require people for cars. If I were an insurance company, I'd ask hard questions. If I were a manager losing my bonus because I was getting charged 2X more per quarter for insurance, I might actually do something about it.

5) Maybe people will start thinking about the systemic issues and how to fix them. An analog is the discussion in the book "Internal Bleeding" on ways to fix systemic problems in hospitals. We certainly need the equivalent for offshore drilling, since the wells aren't going away any time soon.

Hank Roberts said...

I tried to get the 'BP fails booming 101' link and a quote into the Dot.Earth comments, but didn't take out enough bad words and it got rejected. I've tried again. It really illustrates that we _do_have_ people who know how to do a good job available. And that they don't make niceynice and might offend someone while telling the truth.

The people who I worry about are the ones kept insulated from that kind of blunt opinion.

Hank Roberts said...

I tried to get the 'BP fails booming 101' link and a quote into the Dot.Earth comments, but didn't take out enough bad words and it got rejected. I've tried again. It really illustrates that we _do_have_ people who know how to do a good job available. And that they don't make niceynice and might offend someone while telling the truth.

The people who I worry about are the ones kept insulated from that kind of blunt opinion.

Hank Roberts said...

Sorry, double post somehow.

New reply:
> maybe they should be required
> to carry insurance

Note there was a lot of fast money made _selling_ insurance on credit defaults. Bank A agreed to insure bank B, bank B insured bank C, and bank C insured both A and B. "Hey, what could go wrong? -- AIG"

It's not just insurance that's needed. It's responsibility and clawbacks of profit that wasn't real over time.

Hank Roberts said...,0,1634979.story
“Douglas Brown, the rig’s chief mechanic, testified that three officials for rig owner Transocean Ltd. balked at the desire of a BP “company man” to remove drilling mud from the pipe connecting the rig with the well.
That wasn’t Obama. That was the workers who knew what wasn’t safe and got overruled.

If Obama had interrupted their meeting and told them how it should be done, people would have started calling him a socialist or worse.

Oh, wait ....

What will they call him if he starts _listening_ to the workers who know what’s safe, instead of to management? And says the workers ought to decide on safe practices?

Right, they’ll call him a socialist.

John Mashey said...

Hank: Credit Default thingies weren't insurance in the normal sense practiced by real insurance companies. To see what they were really were, read Michael Lewis (of "Liar's Poker" fame) new book, "the Big Short."

It bears *zero* resemblance to normal insurance.

Clawbacks are nice, but I'm familiar with them [they happen around here in VC-land], and I'm afraid the deterrent effect is not as strong as one might think.

Look, this is like crime ... or even getting parking tickets. The certainty of getting caught is far more effective than the penalty, as people can always fool themselves. Again, whatever the incentives, they must be *pervasive* down to the lowest organizational level, and they must *prevent* problems, not just promise a chance that somebody will get zinged later, maybe.

silburnl said...

"It bears *zero* resemblance to normal insurance"

Indeed, specifically the absence of any requirement to have an 'insurable interest' seems to me to have been a key factor in the whole mess.

Also the massive contractual superstructure that built up relative to the value of the underlying instruments that were at risk of default would appear to violate the principle of indemnity.

You can maybe, maybe excuse the quants at JPMorgan who invented CDOs for not having this sort of background knowledge but WTF were the people at AIG and the monolines thinking? They were supposed to be insurance people so 'insurable interest' should have been drilled into them from day 1.

FFS I'm just an IT bod who has picked up bits and pieces of insurance lingo from clients I've worked for and I know about this stuff.


gravityloss said...

Reading the comments, this reminds me of Challenger and Columbia too. The experts knew that problems would be probable, but the organization was built so that they would have had to fight extra hard with dire consequences for themselves to be heard.
It's a tough job to build an organization that is not stifled by doubts and can act but still can stop what it's doing when there is a danger.
Maybe the current NASA "Shuttleland" actually is quite advanced in that regard and a lot could be learned from it. Many inside NASA agree that the STS folks are doing good job. And this is not obvious, it has been built on purpose because it had to be done. So, maybe ask some guru like Wayne Hale, who was formerly a flight director and space shuttle program manager.

Sinc Columbia, there have been disagreements by engineers that have propagated to higher levels, have been heard and examined closer and have turned out not to be reasons for stopping a launch. But there has been at least some actual functioning of the "let's actually listen to concerns and do something" process.

Wayne has blog but glancing at it, lately it's been about specific spaceflight stuff and not something general like that.

Hugh said...

In case anyone had forgotten about what starts next month, here's some more good news on the exposure/vulnerability of the Gulf Coast

John Mashey said...

Luke: yep, and again, I recommend "The Big Short", because it might be worse than you think.

Again, to clarify, my emphasis was not on the insurance-company aspect, as that's only one of the ways. After all, nuclear submarines have been quite reliable without insurance companies, as far as I know, so there are plenty of other mechanisms.

The real issue is one of corporate culture and incentives that work right and cover everybody who needs to.

A reason I suggest insurance companies is that they are used to pricing risk, and the fact that money actually changes hands makes people notice. Similar effects can be achieved if there is, for example, an internal organization with serious power to worry about this.

(This is somewhat analogous to the common practice in professional software engineering organizations of having a Q/A and Release Control organization separate from developers ... and formal procedures such that the Q/A manager can hold a release.
You can tell whether or not somebody means it if the Q/A manager can say: "This release isn't yet reliable enough, and yes, I know we will miss our revenue target, but so be it..." and the CEO backs them up.

The real issue is that one must design human systems to get *all* the right people to do the right things. that isn't easy, but there are existence proofs.

David B. Benson said...


That is, nationalize BP.

That'll certainly put investors on notice of risk.

Hank Roberts said...

Ask also, why do we need all this oil? Why is fuel economy still so bad? A better safer lighter higher-economy car available 30 years ago.

Hat tip to MetaFilter for this.

"... Test drives scored about 32 miles to the gallon, but test crashes suggested passengers might walk away from most crashes up to 50 mph with minimal injuries....
..... NHTSA chief Joan Claybrook was ready to press on in 1980 with a new generation of safety vehicles, setting a target of a 2000-lb. car that could seat four and pass a battery of 40-mph crash tests.

All that ended in January 1981, when the "Morning in America" team from the Reagan administration halted the RSV work ...."