Tuesday, June 2, 2020

Pitfalls in Epidemiological Modelling


Tyler Cowen is sounding just a bit fed up with Twitter, in this Marginal Revolution post and in particular with the way Twitter has degenerated into a name calling equilibrium. Prisoners Dilemma with everyone choosing to betray (what’s the payoff structure that would support such an outcome?).  the point at issue is the explanation which one modeller has given for why his model overestimated the impact of COVID in Germany – his casual reference to Dark Matter prompted people to say that “Dark Matter” is shorthand for “my model doesn’t work, and I don’t know why”.

The sad thing is that if he’d said that it must mean that people were doing more social distancing than he’d anticipated, he would have placed himself squarely in the mainstream of COVID modelling.  At some point we’ll have to face up to the fact that the epi models for COVID have bee pretty much useless.  Most of the world fell into line behind the Imperial College London (ICL) model, despite its tendency to produce massively wide prediction bounds (you can’t call them confidence intervals since most modellers don’t assign probability weights to them).  The Babylon Bee, a satirical website, ran a headline a while ago saying that: “Latest Computer Model Predicts Between 0 And12.6 Billion New COVID-19 Deaths By Summer”  but when it comes to epi modelling, reality is well on its way to overtaking satire. 


The best source for careful discussion of epidemiological  modelling issues is Freddy Sayers’ interviews at UnHerd.  Sayers has interviewed well informed modellers on all sides of the question.  (One of his interviews, with Karol Sikora, was briefly taken down by YouTube because Sikora disagreed with statements by groups like WHO.  Sikora was too optimistic for their taste.  I gather it’s back up now.) 

All of the interviews  are worth watching   , but the one with  Prof. Sunetra Gupta is particularly relevant for issues pertaining to epidemiological modelling and forecasting.  Her point is that SEIR models like the ICL one are underidentified – in effect they’re trying to solve two unknowns with one equation. 

Incidentally, with regard to her point about the possibility that the coronavirus was around in the UK and elsewhere well before the recognized outbreak of the pandemic in those places, there’s increasing evidence that she’s right; on the other hand it doesn’t seem to have spread anything like as quickly as the Oxford version of the model requires. 

The SEIR model (Susceptible, Exposed, Infected, Removed) is theoretically unexceptional.  The issue is the lack of data on the parameter values which have to be plugged into it for simulation purposes.  Since the data which came out of China early in the pandemic proved completely unreliable ( almost as unreliable, it seems, as Chinese data on coffee sales), modellers were forced to take their parameter values from previous epidemics which involved different viruses.  Thus the decision to close elementary schools was based on the assumption that this coronavirus would spread the same way influenza spread, whereas it turned out that this virus’s spread pattern is quite different.

So when you hear someone saying that policy is being determined by science, remember what one British epidemiologist said a couple of weeks ago: on this virus there is no science.  And if you want any further evidence on that, take a look, in order, here and here and keep in the back of your mind another Babylon Bee headline:   “ Florida Ruled To Be In Violation Of Science For Not Having More People Die” 

Meanwhile on a somewhat ghoulish data source note, this guy has probably given New Brunswick’s modellers (led, incidentally, by Ted McDonald of the Economics Department at the University of New Brunswick) the best data set on infectiousness in the country.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.