It is time to stop quivering in our boots in pointless fear of the future and just roll up our sleeves and build it.
- Ray Pierrehumbert

Thursday, May 6, 2010

Abdication of Human Control

Something about Thursday's Wall Street fiasco seems to me to have some wider implications about our predicament. We are running the whole world on autopilot. Everything is fine, at least until a pilot is needed. But then, there isn't one, or if there is one, she doesn't have the right authentication to take control.

Look, we set these things up. Anything they do to us is our fault. A little foresight is needed for the cases where something does go just a tiny bit out of kilter.

Bruce Stirling tweets: Hey Adam Smith, your "invisible hand of the market" has become a crazed robot crushing everything it can grip

and: A ten percent market dip on a typo makes bank-torching Greek rioters look like conservatives



Dan Olner said...

It's not the invisible hand, though, is it? It's a quite specific bunch of traders. Can someone explain to me how this is still being allowed to happen?

Martin said...


King of the Road said...

@ Dan Olner
Sure, easy. It's the Golden Rule.

S said...

Surely there are computer model-based risk assessments for this sort of thing. Why do we only hear the explanation a posteriori? mt says we set it up -- shouldn't some bright economist come along and show everyone how damagingly unstable this is and propose a way to dampen wild swings? Who profits by refusing to build in some automatic safety features?

Tom said...

And the lesson we should take regarding science and its move away from observation and towards meta studies performed from behind a computer is...?

Michael Tobis said...

Tom, Tom, Tom.

I'm sorry but I have to say it again. You're really out of your depth. You have much less of an idea of what you are talking about than you think you do.

My job description is I sit at a computer, type, move a mouse, and click. So is yours. So is the person trading securities and the person who wrote the code the other person uses. Do you really think we are all doing the same thing?

rustneversleeps said...

We'll never get the genie back in the bottle, but all of these automated trading systems, etc., are at least in theory supposed to provide some benefit in terms of improved liquidity, price discovery, etc.

But there is inevitably some cost, and just like everything else it is surely subject to "diminishing returns". I'm sure it could be shown that we got >80% of the "liquidity" benefit from changes that were made 20+ years ago, and we could now be incurring a net cost with further advancements. I think yesterday, and the 2nd- and 3rd-or-more order derivatives on mortgages are both good examples.

It's remarkable how insightful Buffett is on all these things. I think he has stated that society would be better off if all trading of financial assets was limited to a few hours a year or month. Like I opened, that ship has long since sailed, but I think he knows whereof he speaks.

I think the "complexity theorists" should be listened to a great deal more.

@ Tom, thanks for the laugh.

David B. Benson said...

Henry Petroski in his To Enginerr already stated the matter rather nicely.

Another example of not closing the barn door until after ...

Tom said...

Ah, back to that again. Wrong, Michael. Wrong every time you repeat it. But I'm sure it makes you feel better.

Frank Bi said...


Eh? From what I gather, the 10% market diarrhoea was ultimately caused by human error. Though you can say that the huge army of automatic gizmos out there was what caused the error to become this big.

Machines are pretty stupid creatures, but whatever stupid things they can do, they tend to do them quite reliably (a possible exception being the acceleration in Toyota cars). Humans can think at the high level, but they can also make some really bad mistakes. I expect we'll be grappling with this issue -- how best to interface man and machine -- for quite a long time to come.

rustneversleeps said...

the careful allocation of capital... behold! Not sure whether to laugh or cry.

h/t Kedrosky, who says: "a guy conducting a webinar on day-trading goes Full Walter Sobchak as the market (briefly) crumbles"

Horatio Algeranon said...

They said "A Crash
"Will not occur,"
Though Brooksley Born
Did beg to differ.

They said "A Spill
"Will not transpire,"
Though NOAA warned
"It could be dire".

They say "Global Warming
"Is not of concern,"
And the scientists wonder,
"Will we ever learn?"

--The Future Of Denial