A Truly Bad Study
Imagine this study: An academic who is a strong Democrat wants to do a study to discover if Republicans suffer from a psychological tendency to bizarre conspiracy theories. OK, the reasonable mind would already be worried about this. The academic says his methodology will be an online survey of the first 1000 people who reply to him from the comment sections of certain blogs. This is obviously terrible -- a 12-year-old today understands the problems with such online surveys. But the best part is that he advertises the survey only on left-wing sites like the Daily Kos, telling anyone from those heavily Democratic sites that if they self-identify as Republicans, they can take this survey and their survey responses will be published as typical of Republicans. Anyone predict what he would get?
It is hard to believe that even in this post-modern academic world, that such a piece of garbage could get published. But it did. The only difference is that the academic was a strong believer in global warming, he was writing about skeptics, and sought out survey respondents only on strong-believer sites. What makes this story particularly delicious is the juxtaposition of the author's self-appointed role as defender of science with his atrocious scientific methodology. The whole story is simply amazing, and you can read about it at JoNova's site.
In one way, it is appropriate to have this published in a psychology journal, as it is such a great example of the psychological need for confirmation. You can just see those climate alarmists breathing a little easier - "we don't have to listen to those guys, do we?" No need for debate, no need for analysis, no need for thought. Just immediate dismissal of their arguments because they come from, well, bad people. Argumentum ad hominem, indeed.
GilM:
I'm sure somebody will use this example in an examination of how motivated reasoning causes alarmists to accept bad science.
August 30, 2012, 11:08 amThis is so bad that I was briefly wondering if the author was actually a skeptic trying discredit the alarmist community. But, that may be too much to hope for.
lucia liljegren:
I am waiting to see how long he will continue to fail to reveal the 5 skeptic blogs he supposedly contacted.
August 30, 2012, 1:25 pmMingoV:
What is truly horrific about this debacle is that "Psychological Science" is one of the top peer-reviewed journals in its field. Then again, the entire field of psychology has problems differentiating good studies from bad.
August 30, 2012, 2:28 pm