(If you’re not already following him, I highly recommend reading Luca Trevisan’s dispatches from Milan, much more interesting than what I write below.)
On the topic of my last post, Ross Douthat writes in the New York Times that “In the fog of coronavirus, there are no experts”, even citing Scott Aaronson’s post. Both Aaronson and Douthat make the point that the COVID-19 crisis is so surprising and unprecedented, and experts were so much in the dark, that there is no reason to trust them over non expert “common sense” or “armchair epidemiologists”.
It’s true that the “expert models” have significant uncertainty, hardwired constants, noisy data, and dubious assumptions. It is also true that many countries (especially those that didn’t learn from the 2003 SARS epidemic) bungled their initial response. But do we really need to challenge the notion of expertise itself? To what extent was this pandemic not predicted by experts or progressed in ways defying their expectations?
Here is what some of these experts and institutions were saying in the recent past:
“The thing I’m most concerned about … is the emergence of a new virus that the body doesn’t have any background experience with, that is very transmissible, highly transmissible from person to person, and has a high degree of morbidity and mortality … a respiratory illness that can spread even before someone is so sick that you want to keep them in bed.” Dr. Anthoni Fauci, 2019.
“High-impact respiratory pathogens … pose particular global risks … [they] are spread via respiratory droplets; they can infect a large number of people very quickly and with today’s transportation infrastructure, move rapidly across multiple geographies. … There is insufficient R&D investment and planning for innovative vaccine development and manufacture, broad-spectrum antivirals, … In addition, such a pandemic requires advance planning across multiple sectors … Epidemic control costs would completely overwhelm the current financing arrangements for emergency response.” WHO world at risk report, 2019.
“respiratory transmission …. is the transmission route posing the greatest pandemic risk … [since it] can occur with coughing or simply breathing (in aerosol transmission), making containment much more challenging. … If a pathogen is capable of causing asymptomatic or mildly symptomatic infections that either do not or only minimally interrupt activities of daily living, many individuals may be exposed. Viruses that cause the common cold, including coronaviruses, have this attribute.” JHU report, 2019.
As an experiment, I also tried to compare the response of experts and “contrarians” in real time as the novel coronavirus was discovered, trying to see if it’s really the case that, as Douthat says, “up until mid-March you were better off trusting the alarmists of anonymous Twitter than the official pronouncements from the custodians of public health”. I chose both experts and contrarians that are active on Twitter. I was initially planning to look at several people but due to laziness am just taking Imperial college’s J-IDEA institute for the expert, and Robin Hanson for the contrarian. I also looked at Douthat’s twitter feed, to see if he followed his own advice. Initially I thought I would go all the way to March but have no time so just looked at the period from January 1 till February 14th. I leave any conclusions to the reader.
January 1-19:
(Context: novel coronavirus confirmed in Wuhan, initially unclear if there is human to human transmission – this was confirmed by China on January 20 though suspected before.)
Here is one of several tweets by Imperial from this period:
I didn’t see any tweets from Hanson or Douthat on this topic.
January 20-31:
(Context: first confirmed cases in several countries, including the US, WHO declares emergency in Jan 30, US restricts travel from China on Jan 31. By then there are about 10K confirmed cases and 213 deaths worldwide.)
On January 25th Imperial college estimated the novel coronavirus “R0” parameter as 2.6:
Hanson tweeted approvingly about China’s response and that this situation might help the “more authoritarian” U.S. presidential candidate:
Still no tweet from Douthat on this topic though he did say in January 29th that compared to issues in the past the U.S.’s problems in the 2020’s are “problem of decadence” rather than any crisis like the late 1970’s:
February 1-14:
(Context: Diamond princess cruise ship quaranteed, disease gets COVID-19 official name, first death in Europe)
Imperial continues to tweet extensively, including the following early estimates of the case fatality rates:
Robin Hanson correctly realizes this is going to spread wide:
Hanson tweets quite a lot about this, including potential social implications. Up to February 13th there is nothing too “contrarian” at this point, but also no information that could not be gotten from the experts:
In February 14 Hansons makes a very contrarian position when he proposes “controlled infection” as a solution:
To the anticipated “you first” objection he responds “I proposed compensating volunteers via cash or medical priority for associates, & I’d seriously consider such offers.”. He doesn’t mention that he is much less strapped for cash than some of the would be “volunteers”.
Still no tweet from Douthat about COVID-19 though he does write that we live in an “age of decadence”:
Windows On Theory 
NN Taleb was the most accurate as usual. He tweeted a position paper around 5th Jan, warning of a global pandemic, and blasting health experts for playing it down.
Do you have a link? Tried to search for all his tweets from January 1 till 24 and didn’t find anything related to the Coronavirus. Could be that I missed it since I’m doing it on my phone.
The only thing I found is a position paper from January 26 which is curiously almost detail free – seems like it could have been written as is about SARS , H1N1 or any other infectious disease
https://necsi.edu/systemic-risk-of-pandemic-via-novel-pathogens-coronavirus-a-note
At this point there were much more informative reports from Imperial (am sure others too, but can’t look right now).
Well, someone at Bridgewater (“World’s largest hedge fund”) made, in **November**, a “$1.5 Billion Options Bet on Falling Market”. The publicatiion on “The Wall Street Journal” is from November 22-nd, here it is
https://www.wsj.com/articles/bridgewater-bets-big-on-market-drop-11574418601
citation: “Bridgewater Associates LP has bet more than $1 billion that stock markets around the world will fall by March, said people familiar with the matter.”
I wonder what the experts at Bridgewater responsible for the investment knew about, and from where. Or maybe they just make such bets on a regular basis?
(citation from Guardian: “Earliest case detected on 17 November, weeks before authorities acknowledged new virus, says Chinese media” https://www.theguardian.com/world/2020/mar/13/first-covid-19-case-happened-in-november-china-government-records-show-report)
Two points:
1) I think it was generally believed that the world economy is “due” for a recession so in its own the Bridgewater bet is not that surprising.
2) Yes apparently the US intelligence was monitoring the situation since late November and it was included in briefings https://www.forbes.com/sites/mattperez/2020/04/08/report-us-intelligence-officials-warned-about-coronavirus-in-wuhan-in-late-november/
Makes you wonder why pay for the world’s best intelligence if you will not act on anything they say until it reaches Fox News.
But at the same time, I think the ABC news article is not satisfying because it is not clear if the evidence they presented was actionable even if the higher ups could judge it obectively (a big assumption). Like what tangible evidence they had for that prediction (in the sense of could this have been written about any other disease). A report that just says ‘a pandemic will be a cataclysmic event’ is not actionable in my opinion.
I wonder if the report about November is accurate or they got it off by a month – after al as far as we know “Patient zero” may have been Wei Guixian who only fell ill on December 10.
I don’t think this changes much for the current discussion though – the US still acted much much later
Harry H: I don’t think that there was radical action to be taken in late November, but the options are not between shutting down the economy and doing nothing. This information could definitely have been shared with the CDC and other agencies, and the administration could have asked them to report how prepared they are. You are absolutely right that this could have been written about any disease, but it’s also the case that there are preparations that can and should be taken for any disease.
To get a sense of proportion – the question is not just why the US didn’t act in late November or December when they had private early information, but why we didn’t do much over January, February and early March. Indeed, in March 6 – three months (!) after these reports – president Trump referred to the novel coronavirus as an “unforseen problem” that “came out of nowhere” ( https://www.whitehouse.gov/briefings-statements/remarks-president-trump-signing-coronavirus-preparedness-response-supplemental-appropriations-act-2020/ ).
Btw I don’t need to try to invent suggestions as to what the federal government could have been doing in early stages of a pandemic. It turns out they have a 69 page playbook for it. They just didn’t follow it:
https://www.politico.com/news/2020/03/25/trump-coronavirus-national-security-council-149285
I am not sure that the discussion over at the Marginal Revolution blog adds something very significant to the debate about “experts”, however, I see it as a general feeling that many people seem unsatisfied about their claims. Link here: https://marginalrevolution.com/marginalrevolution/2020/04/what-does-this-economist-think-of-epidemiology.html
Disclaimer: I am not fully endorsing it. Simply, it seems to me that the situation is much more nuanced than what Boaz seems to be thinking (if I understood correctly his point).
I didn’t read the full discussion comments but not very impressed by questions in the original blog post such as “As a class of scientists, how much are epidemiologists paid? Is good or bad news better for their salaries?” or “How smart are they? What are their average GRE scores?”.
Even predicting elections, which happen on a regular basis, and where “testing” is as easy as calling someone on the phone and asking them a question, is not an easy task. Predicting the progression of a novel epidemic is much harder, this depend on both extracting the parameters of the virus itself from highly noisy data (as well as genetic analysis), but also on predicting the response of societies to the progression of the disease.
Note that there are several different ways of making these predictions, some of them involve trying to fit the curve of progressions in past cases, while others are mechanistic models that simulate every person. As I said, I believe most of those involve hardwired constants and dubious assumptions. I have no doubt that these models can be improved, and indeed will be improved eventually, through collaborations of people with different expertise.
More generally, the questions such as “have they read Tetlock?” or “how good are their predictions” seem to reduce epidemiology to Nate Silver like attempts to predict the topline numbers.
Not that the topline numbers are not very important at the phase we are in, but if you look at reports such as https://apps.who.int/gpmb/assets/thematic_papers/tr-6.pdf
there is much work and thought of trying to think in advance how local, national and international health system can better prepare for such a pandemic (which was predictable and indeed even predicted) rather than trying to be the person that was most correct in guessing how many people died.
Thank you. I read the September report you linked. However, please let me say that I am not very much impressed, either. They were epidemiologists, right? What else should have been doing? What I am trying to say is that there were experts who gave the terrible suggestion to do nothing in the UK at the start of the pandemics, (just to back off later on..) and many people have died because of that. The rationale was to create herd immunity, when there is not a single evidence that immunity to the Covid19 exists or not, and if it exists nobody really knows how long it lasts. Same mess in Sweden (always initially supported by the so-called “evidence”, only to be changed later on..). Same in Italy. Please read the following interview (in Italian, but DeepL does a reasonable job) https://themeticulous.altervista.org/blog/lintervista-errori-di-ricciardi-e-oms-perso-un-mese-sugli-asintomatici/
The same WHO has made several questionable statements (to say the least..) about the usefulness of mass testing, the usefulness of wearing masks, etc. Maybe in the US the situation is different, I do not know, and incompetence is in other echelons…
The way I see it that at a very high level we have a situation that people in the field predicted has a decent likelihood of happening. There was also a scientific consensus on the right approaches to prepare for this – testing and information sharing infrastructure, personal protective equipment and facilities to treat highly infectious patients in hospitals, infrastructure for rapid vaccine development, etc.. They also made the correct argument that while these things are costly, these costs would pale on comparison with the economic impact of such a pandemic. It seems that generally the countries that had the experience with SARS and prepared their institutions better did better. These countries didn’t ask random bloggers what to do, but rather followed the advice of experts and the general scientific community.
I don’t know who is in charge of the response in each country, what are their qualifications, and how they are appointed, and so I would definitely not vouch for all of them. It could be that some of these people were incompetent. If so, the solution would have been to appoint a competent person in charge, of which there are many in the field of epidemiology. The solution would not have been to make a random blogger in charge. Similarly, the solution to the 2008 recession would not have been to make the chair of the Federal reserve an epidemiologist.
Even competent people will make mistakes in a fast moving situation, whether it is war, a pandemic, or even a sports game. There will always be the case that an expert said X, some random blogger said Y, and in retrospect the random blogger was right. That doesn’t mean we should make this blogger a general, head of the CDC, or a coach.
Zach Weinersmith, Maggie Koerth, Laura Bronner and Jasmine Mithani do a great job in 538 of summarizing the issues with modeling and making predictions on COVID-19:
https://fivethirtyeight.com/features/a-comic-strip-tour-of-the-wild-world-of-pandemic-modeling/