Coyote Blog

Some thoughts on the obsession with peer review as the gold standard guarantee of climate science goodness, from Climate Skeptic:

One of the weird aspects of climate science is the over-emphasis on peer
review as the ne plus ultra guarantor of believable results.  This is absurd. 
At best, peer review is a screen for whether a study is worthy of occupying
limited publication space, not for whether it is correct.  Peer review, again at
best, focuses on whether a study has some minimum level of rigor and coherence
and whether it offers up findings that are new or somehow advance the ball on an
important topic. 

In "big
boy sciences" like physics, study findings are not considered vetted simply
because they are peer-reviewed.  They are vetted only after numerous other
scientists have been able to replicate the results, or have at least failed to
tear the original results down.  Often, this vetting process is undertaken by
people who may even be openly hostile to the original study group.  For some
reason, climate scientists cry foul when this occurs in their profession, but
mathematicians and physicists accept it, because they know that findings need to
be able to survive the scrutiny of enemies, not just of friends.  To this end,
an important part of peer review is to make sure the publication of the study
includes all the detail on methodology and data that others might need to
replicate the results  (which is something climate reviewers are particularly bad at).

In fact, there are good arguments to be made that strong peer review may even
be counter-productive to scientific advancement.  The reason is that peer
review, by the nature of human beings and the incentives they tend to have, is
often inherently conservative.  Studies that produce results the community
expects often receive only cursory scrutiny doled out by insiders chummy with
the authors.  Studies that show wildly unexpected results sometimes have trouble
getting published at all.

 As I read this, it strikes me that one way to describe
climate is that it acts like a social science, like sociology or gender studies,
rather than like a physical science.  I will ahve to think about this -- it
would be an interesting hypothesis to expand on in more depth.  Some quick
parallels of why I think it is more like a social science:

• Bad statistical methodology  (a hallmark, unfortunately, of much of social
  science)
• Emphasis on peer review over replication
• Reliance on computer models rather than observation
• Belief there is a "right" answer for society with subsequent bias to study
  results towards that answer  (example,
  and another
  example)