The only thing we can be sure of about the future is that it will be absolutely fantastic. So if what I say now seems to you to be very reasonable, then I have failed completely. Only if what I tell you appears absolutely unbelievable, have we any chance of visualizing the future as it really will happen.

- Arthur C. Clarke (h/t Brin)

Thursday, September 4, 2008

The Dunning-Kruger Effect

Remember this?
The more you know the less you care -- at least that seems to be the case with global warming. A telephone survey of 1,093 Americans by two Texas A&M University political scientists and a former colleague indicates that trend, as explained in their recent article in the peer-reviewed journal Risk Analysis.
It turned out, as John Mashey noticed, that "knowledge" in the study was self-assessed by the subjects. The real negative correlation is between how much you care and how much you THINK you know, a very different measure.

Via a comment on the Python list, I stumbled across something on Wikipedia called the Dunning-Kruger effect that would indicate that this wasn't an unusual outcome at all.
Kruger and Dunning noted a number of previous studies which tend to suggest that in skills as diverse as reading comprehension, operating a motor vehicle, and playing chess or tennis, "ignorance more frequently begets confidence than does knowledge" (as Charles Darwin put it). They hypothesized that with a typical skill which humans may possess in greater or lesser degree,

- Incompetent individuals tend to overestimate their own level of skill.

- Incompetent individuals fail to recognize genuine skill in others.

- Incompetent individuals fail to recognize the extremity of their inadequacy.

- If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill.

...

Meanwhile, people with true knowledge tended to underestimate their competence.

H/T to Anna Haynes who reminded me of all this in a recent email exchange.

7 comments:

John Mashey said...

DKE is indeed a useful model.
it seems a contributory effect:

Google: dunning kruger mashey

Hank Roberts said...

The original:

http://www.apa.org/journals/features/psp7761121.pdf

Cited by over 400 papers:

http://scholar.google.com/scholar?hl=en&lr=&cites=4155734925719668623&um=1&ie=UTF-8&sa=X&oi=science_links&resnum=1&ct=sl-citedby

Hank Roberts said...

PS, seriously, have a look at some of the citing studies including subsequent work by the same author(s), which addresses some of the critiques.

This would rather well explain some of the vehement "this can't be possible" arguments-from-suspicion.

E.g.:

"Conclusions
The main goal of this research was to determine which error led people to feel "holier than thou." We found that, in the moral domain at least, people were much less accurate when predicting their own behavior than when predicting others'. Participants consistently, and grossly, overestimated the likelihood that they themselves would act in a selfless and altruistic manner, whereas the predictions made about others tended to converge more closely with reality...."

http://gsbwww.uchicago.edu/fac/nicholas.epley/EpleyandDunning2001.pdf

EliRabett said...

It's well known that if you ask students who knows their algebra, the ones who say they do, don't. Fairly useful in identifying those who need help

Marion Delgado said...

There is also the idea that the more you genuinely know the more areas of ignorance you encounter.

And of course, greater knowledge can be paralyzing depending on its focus and the initial conditions.

But as hank hints, the main reaction to a dunning kruger effect situation is personal attacks on the presenter and their standing to assess competency.

Marion Delgado said...

Also, a propos de rien, has anyone else read Penrose's Road to Reality working the examples as you go?

I found it enlightening because I had studied virtually everything in it at one time or another but actually doing exercises showed I remembered less than half of what I knew (since I mostly have worked in news and programming since I last studied science).

It may seem obvious, but submitting to an arbitrary external norm of some sort is the only remedy I know of for this "effect." Cf. again and again, Feynman's "and you are the easiest person to fool."

bernie said...

eli:
You can't possibly be serious. Now if you have good reson to believe that x% know their algebra and 10*x% say they do - then you have a pretty good chance (90%) chance of being correct. However, if 1.1*x% say they know their algebra then you clearly will insult a lot of the folks who actually do know their stuff.
You need to know or be pretty certain of the relative numbers of who really know their stuff against those who say they do before you can make such an assertion.

P.S. My algebra is rusty