Coyote Blog

In this post, I wrote about parallels between climate and COVID alarm and related issues of computer modelling.  I realized I left out at least one parallel.

In the world of climate, computer model results are often used as the counterfactual case.  Let me give you an example.  The world has warmed over the last 100 years at the same time atmospheric CO2 concentration has increased.  Obviously, to truly judge the effect of CO2 on temperatures, we would like to know what the temperatures would have been over the last 100 years without rising CO2 concentrations.  But we don't have thermometers that read "with" and "without" CO2.

I remember I got caught up in this years ago when I published an analysis that showed that estimates of temperature sensitivity to CO2 concentrations used in projections going forward greatly over-predicted the amount of warming we have seen already.  In other words, there had not been enough warming historically to justify such high sensitivity numbers.  In response, I was told that alarmists considered the base case without CO2 increases to be a cooling world, because that is what some models showed.  Compared to this cooling counterfactual, they argued that the warming from CO2 historically had been much higher.

By the way, this argument always gets to be very circular.  When you really dig into the assumptions of the counter-factual models, they are based on assumptions that temperature sensitivity to CO2 is high.  Thus models predicated on high sensitivity are used to justify the assumption of high sensitivity.

I thought of all this today when I saw this post on COVID models and interventions from Kevin Drum.  I read Drum because, though I don't love his politics, he is more likely than most team-politics writers from either the Coke or Pepsi party to do a reasonable job of data analysis and interpretation.  But I have to fault him for this post, which I think is just terrible.  You can click through to see the chart but here is the text:

At the end of March, the highest estimate for [NY State] hospitalizations was 136,000+. Today the peak is estimated at about 30,000. That’s a difference of 5x. Did the modelers screw up?

Not really. Remember the Imperial College projections for the United States? They estimated about 2 million deaths if nothing was done; 1 million deaths if some countermeasures were taken; and 200,000 deaths if stringent countermeasures were taken. That’s a range of 10x. If you figure that we’ve taken fairly stringent countermeasures but not the maximum possible, then a reduction of 5x is about what you’d expect. Alternatively, if you ignore the Columbia University projection as an outlier, the IHME estimate has only gone down by about 2x. That’s what you’d expect if we took countermeasures that were just a little more stringent than their model assumed.

At the end of March it was still not clear how stringent and how effective the coronavirus countermeasures would be. In the event, it looks like they worked pretty well, cutting cases by at least 2x and possibly more. This is why the model estimates have gone down: because we followed expert advice and locked ourselves down. Just as we hoped.

Treating the early model estimates as if they are accurate representations of the "no intervention" counter-factual is just absurd.   It is particularly absurd in this case as he actually quotes a model -- the early Imperial College model -- that is demonstrably grossly flawed.  He is positing that we are in the Imperial College  middle intervention case, which estimated a million deaths in the US and is likely to be off by more than an order of magnitude.  Given this clear model/estimate miss, why in the world does he treat early Columbia and McKinsey models as accurate representations of the counter-factual?  Isn't it at least as likely that these models were just as flawed as the Imperial College models (and for many of the same reasons)?

The way he uses the IHME model results is also  flawed.  He acts like the reductions in the IHME estimates are due to countermeasures, but IHME has always assumed full counter-measures so it is impossible to use the numbers the way he wants to use them.

So I finally had a day or two of downtime from trying to keep my business afloat (it's weird reading all the internet memes of people at home bored when I have never been busier).  I wondered why I was initially, and remain, skeptical of apocalyptic COVID-19 projections.

I have been skeptical about extreme global warming and climate change forecasts, but those were informed by my knowledge of physics and dynamic systems (e.g. feedback mechanics).  I have been immensely skeptical of Elon Musk, but again that skepticism has been informed by domain knowledge (e.g. engineering in the case of the hyperloop and business strategy in the case of SolarCity and Tesla).  But I have no domain knowledge that is at all relevant to disease transfer and pathology.  So why was I immediately skeptical when, for example, the governor of Texas was told by "experts" that a million persons would die in Texas if a lock-down order was not issued?

I think the reason for my skepticism was pattern recognition -- I saw a lot of elements in COVID-19 modelling and responses that appeared really similar to what I thought were the most questionable aspects of climate science.  For example:

• We seem to have a sorting process of "experts" that selects for only the most extreme.  We start any such question, such as forecasting disease death rates or global temperature increases, with a wide range of opinion among people with domain knowledge.  When presented with a range of possible outcomes, the media's incentives generally push it to present the most extreme.  So if five folks say 100,000 might die and one person says a million, the media will feature the latter person as their "expert" and tell the public "up to a million expected to die."  After this new "expert" is repetitively featured in the media, that person becomes the go-to expert for politicians, as politicians want to be seen by the public to be using "experts" the public recognizes as "experts."
• Computer models are converted from tools to project out the implications of a certain set of starting hypotheses and assumptions into "facts" in and of themselves.   They are treated as having a reality, and a certainty, that actually exceeds that of their inputs (a scientific absurdity but a media reality I have observed so many times I gave it the name "data-washing").  Never are the key assumptions that drive the model's behavior ever disclosed along with the model results.  Rather than go on forever on this topic, I will refer you to my earlier article.
• Defenders of alarmist projections cloak themselves in a mantle of being pro-science.  Their discussions of the topic tend to by science-y without being scientific.  They tend to understand one aspect of the science -- exponential growth in viruses or tipping points in systems dominated by positive feedback.  But they don't really understand it -- for example, what is interesting about exponential growth is not the math of its growth, but what stops the growth from being infinite.  Why doesn't a bacteria culture grow to the mass of the Earth, or nuclear fission continue until all the fuel is used up?  We are going to have a lot of problem with this after COVID-19.  People will want to attribute the end of the exponential growth to lock-downs and distancing, but it's hard to really make this analysis without understanding at what point -- and there is a point -- the virus's growth would have turned down anyway.
• Alarmists who claim to be anti-science have a tendency to insist on "solutions" that have absolutely no basis in science, or even ones that science has proven to be utterly bankrupt.  Ethanol and wind power likely do little to reduce CO2 emissions and may make them worse, yet we spend billions on them as taxpayers.  And don't get me started on plastic bag and straw bans.   I am willing to cut COVID-19 responses a little more slack because we don't have the time to do elaborate studies.  But just don't tell me lockdown orders are science -- they are guesses as to the correct response.  I live in Phoenix where it was sunny and 80F this weekend.  We are on lockdown in our houses.  I could argue that ordering everyone out into the natural disinfectant of heat and sunlight for 2 hours a day is as effective a response as forcing families into their houses (initial data, though it is sketchy, of limited transfer of the virus in summertime Australia is interesting -- only a small portion of cases are from community transfer.  By comparison less than a half percent of US cases from travel).
• In both cases, advocates of the precautionary principle seem to rule the day.  I would argue that in practice, the precautionary principle means that any steps that might conceivably limit something bad should be pursued irregardless of cost.  You see a form of this all over social media, which folks arguing that it is wrong to balance deaths against money, and any life spared is worth the cost.  But this is absurd two at least two reasons
  • First, unemployment and economic recession have real, proven effects on mortality.  Shut down the economy to reduce CO2 or virus spread, and people will die
  • Second, if we really followed this principle for everything we would be back in the stone age.  Take the flu.  15,000-20,000 people will die of the flu every year in the US -- my healthy 25-year-old nephew died of the flu.  Are we going to shut down the economy next year in flu season?  It would reduce flu deaths.  Or take the 37,000 people killed each year in the US in motor vehicle accidents.  With the lockdowns, that figure is certainly reduced right now.  Should we just shut down the economy forever, it sure would reduce car fatalities?
• And of course there is the political polarization of what should be scientific opinion.  The Nevada and Michigan governors initially banned chloroquine treatment strategies for no good reason other than the fact that Trump publicly highlighted them as promising.

Update:  Prediction from climate applied to COVID-19:  No one will go back and call out widely-used models for failing to accurately model the disease or attempt to learn from their mistakes.  If it is ever mentioned that these models grossly over-estimated deaths, it will be forgiven as being exaggeration in a good cause.  (Somewhat related, Bryan Caplan on Social Desirability Bias)