Squishy Words That Create Problems For Using Results of Scientific Studies

The IPCC AR4 summary report had this critical conclusion:

Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic GHG concentrations.[7] It is likely that there has been significant anthropogenic warming over the past 50 years averaged over each continent (except Antarctica)

I want to come back to this in a second, but here is a story the Bryan Caplan posted on his blog.  He is quoting from Tetlock and Gardner's Superforecasting

In March 1951 National Intelligence Estimate (NIE) 29-51 was published.  "Although it is impossible to determine which course of action the Kremlin is likely to adopt," the report concluded, "we believe that the extent of [Eastern European] military and propaganda preparations indicate that an attack on Yugoslavia in 1951 should be considered a serious possibility." ...But a few days later, [Sherman] Kent was chatting with a senior State Department official who casually asked, "By the way, what did you people mean by the expression 'serious possibility'?  What kind of odds did you have in mind?"  Kent said he was pessimistic.  He felt the odds were about 65 to 35 in favor of an attack.  The official was started.  He and his colleagues had taken "serious possibility" to mean much lower odds.

Disturbed, Kent went back to his team.  They had all agreed to use "serious possibility" in the NIE so Kent asked each person, in turn, what he thought it meant.  One analyst said it meant odds of about 80 to 20, or four times more likely than not that there would be an invasion.  Another thought it meant odds of 20 to 80 - exactly the opposite.  Other answers were scattered between these extremes.  Kent was floored.

Let's go back to the IPCC summary conclusion, which is quoted and used all over the place  (no one in the media ever actually digs into the charts and analysis, they just stop at this quote).  A few thoughts:

  1. This kind of conclusion is typical of team process and perhaps is a reason that large teams shouldn't do scientific studies.  We wouldn't have aspirin if 500 people all had to agree on a recommendation to allow it.
  2. Climate alarmists often claim "consensus".  Part of the way they get consensus is by excluding anyone who disagrees with them from the IPCC process and publication.  But even within the remaining core, scientists have vast differences in how they evaluate the data.  Consensus only exists because the conclusions use weasel words with uncertain meaning like "most"  and "significant"  (rather than a percentage) and "very likely" (rather than a probability).
  3. Is "most" 51% or 95%?  The difference between these two is almost a doubling of the implied temperature sensitivity to CO2  -- close to the magnitude of difference between lukewarmer and IPCC estimates.  Many skeptics (including myself) think past warming due to man might be 0.3-0.4C which is very nearly encompassed by "most".
  4. It may be that this uncertainty is treated as a feature, not a bug, by activists, who can take a word scientists meant to mean 51% and portray it as meaning nearly 100%.

For an example of this sort of thing taken to an extreme, arguably corrupt level, consider the original 97% global warming consensus survey which asked 77 scientists hand-selected from a pool of over 10,000 working on climate-related topics two questions.  Answering yes to the two questions put you in the 97%.  In the context of what was written above, note the wording:

That anything-but-scientific survey asked two questions. The first: “When compared with pre-1800s levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?”  Few would be expected to dispute this…the planet began thawing out of the “Little Ice Age” in the middle 19th century, predating the Industrial Revolution. (That was the coldest period since the last real Ice Age ended roughly 10,000 years ago.)

The second question asked: “Do you think human activity is a significant contributing factor in changing mean global temperatures?” So what constitutes “significant”? Does “changing” include both cooling and warming… and for both “better” and “worse”? And which contributions…does this include land use changes, such as agriculture and deforestation?

Good Lord, I am a hated skeptic frequently derided as a denier and I would answer both "yes" and be in the 97% consensus.  So would most all of the prominent science-based skeptics you have ever heard of.

 

7 Comments

  1. Richard Harrington:

    97% of all statistics are made up.

  2. pbft:

    In our consulting business we refer to these as 'junk words' and a sign that you need to get actual data. We do an exercise similar to that described above in workshops - have each person write down a number for phrases like 'the box is very heavy' or 'it's too far to the grocery store'. It's a real eye-opener to see the range of values that you'll get.

  3. Jens Fiederer:

    I think the actual quotation was

    "97% of all statistics on the internet are made up."
    - Abraham Lincoln

  4. Zachriel:

    Coyote: The IPCC AR4 summary report had this critical conclusion ... Most ... very likely ... likely

    IPCC defines
    Term, Likelihood of the Outcome
    Virtually certain, 99-100% probability
    Very likely, 90-100% probability
    Likely, 66-100% probability
    About as likely as not, 33 to 66% probability
    Unlikely, 0-33% probability
    Very unlikely, 0-10% probability
    Exceptionally unlikely, 0-1% probability

    "Most" is conventionally defined as more than other possibilities; in this binary case, most means more than half.

    Coyote: The second question asked: “Do you think human activity is a significant contributing factor in changing mean global temperatures?” So what constitutes “significant”? Does “changing” include both cooling and warming… and for both “better” and “worse”? And which contributions…does this include land use changes, such as agriculture and deforestation?

    "Significant" is conventionally defined as of a noticeably or measurably large amount. Changing certainly includes both cooling and warming, though in this case, we can presume the question will be read as net change. "Better or worse" is not part of the question. Anthropogenic contributions certainly would include land use changes, though CO2 seems to be the primary mechanism of anthropogenic global warming.

  5. Q46:

    ‘Likely’, ‘very likely’ = we don’t know, just guessing but want to sound convincing.

    These are words used by fortune tellers not scientists... but then global warming/climate change has propagandists not scientists.

  6. kidmugsy:

    When someone tells me that something is significant I ask what it signifies.

  7. Zachriel:

    In this case, it means that humans are having a non-negligible effect on climate.