Thursday, September 4, 2008

The Dunning-Kruger Effect

Remember this?
The more you know the less you care -- at least that seems to be the case with global warming. A telephone survey of 1,093 Americans by two Texas A&M University political scientists and a former colleague indicates that trend, as explained in their recent article in the peer-reviewed journal Risk Analysis.
It turned out, as John Mashey noticed, that "knowledge" in the study was self-assessed by the subjects. The real negative correlation is between how much you care and how much you THINK you know, a very different measure.

Via a comment on the Python list, I stumbled across something on Wikipedia called the Dunning-Kruger effect that would indicate that this wasn't an unusual outcome at all.
Kruger and Dunning noted a number of previous studies which tend to suggest that in skills as diverse as reading comprehension, operating a motor vehicle, and playing chess or tennis, "ignorance more frequently begets confidence than does knowledge" (as Charles Darwin put it). They hypothesized that with a typical skill which humans may possess in greater or lesser degree,

- Incompetent individuals tend to overestimate their own level of skill.

- Incompetent individuals fail to recognize genuine skill in others.

- Incompetent individuals fail to recognize the extremity of their inadequacy.

- If they can be trained to substantially improve their own skill level, these individuals can recognize and acknowledge their own previous lack of skill.

...

Meanwhile, people with true knowledge tended to underestimate their competence.

H/T to Anna Haynes who reminded me of all this in a recent email exchange.

7 comments:

  1. DKE is indeed a useful model.
    it seems a contributory effect:

    Google: dunning kruger mashey

    ReplyDelete
  2. The original:

    http://www.apa.org/journals/features/psp7761121.pdf

    Cited by over 400 papers:

    http://scholar.google.com/scholar?hl=en&lr=&cites=4155734925719668623&um=1&ie=UTF-8&sa=X&oi=science_links&resnum=1&ct=sl-citedby

    ReplyDelete
  3. PS, seriously, have a look at some of the citing studies including subsequent work by the same author(s), which addresses some of the critiques.

    This would rather well explain some of the vehement "this can't be possible" arguments-from-suspicion.

    E.g.:

    "Conclusions
    The main goal of this research was to determine which error led people to feel "holier than thou." We found that, in the moral domain at least, people were much less accurate when predicting their own behavior than when predicting others'. Participants consistently, and grossly, overestimated the likelihood that they themselves would act in a selfless and altruistic manner, whereas the predictions made about others tended to converge more closely with reality...."

    http://gsbwww.uchicago.edu/fac/nicholas.epley/EpleyandDunning2001.pdf

    ReplyDelete
  4. It's well known that if you ask students who knows their algebra, the ones who say they do, don't. Fairly useful in identifying those who need help

    ReplyDelete
  5. There is also the idea that the more you genuinely know the more areas of ignorance you encounter.

    And of course, greater knowledge can be paralyzing depending on its focus and the initial conditions.

    But as hank hints, the main reaction to a dunning kruger effect situation is personal attacks on the presenter and their standing to assess competency.

    ReplyDelete
  6. Also, a propos de rien, has anyone else read Penrose's Road to Reality working the examples as you go?

    I found it enlightening because I had studied virtually everything in it at one time or another but actually doing exercises showed I remembered less than half of what I knew (since I mostly have worked in news and programming since I last studied science).

    It may seem obvious, but submitting to an arbitrary external norm of some sort is the only remedy I know of for this "effect." Cf. again and again, Feynman's "and you are the easiest person to fool."

    ReplyDelete
  7. eli:
    You can't possibly be serious. Now if you have good reson to believe that x% know their algebra and 10*x% say they do - then you have a pretty good chance (90%) chance of being correct. However, if 1.1*x% say they know their algebra then you clearly will insult a lot of the folks who actually do know their stuff.
    You need to know or be pretty certain of the relative numbers of who really know their stuff against those who say they do before you can make such an assertion.

    P.S. My algebra is rusty

    ReplyDelete

Moderation is on. Apologies for any delays.



Err on the side of politesse and understatement please.



Before you speak, ask yourself if what you have to say will improve on silence.