Monthly Archives: April 2020

by Buzz Hollander MD

Antibody testing for COVID-19 is the new darling of the media. It seems that everyone wants to know whether that little scratch in their throat they had in early February might just mean they have antibodies to SARS-CoV-2! As scientists are interviewed touting the impressive accuracy of these tests, interest continues to grow about antibody testing being the key to “re-opening the economy” as people are able to learn their immune status. But is this going to be as good as it sounds?

Let’s look at two examples to better understand; one hypothetical, the other real world:

Covania is a small town of 1000 people in the outskirts of a metropolitan area with a significant COVID-19 outbreak. 5 Covanians ended up needing to be hospitalized in the initial wave of the outbreak, two required a ventilator, and one died of COVID-19. Officially, 25 Covanians were tested and 10 were found to be positive, including the 5 hospitalized patients. However, in our hypothetical example, 50 Covanians actually had COVID-19, although 10 never exhibited symptoms. The very high test positive rate of 40%, and the rather high testing rate of 2.5% of the population, mirror the numbers for New York City rather well; as does the deaths per million (DPM) of 1000 DPM that Covania’s single death produces. Covania’s true prevalence of COVID-19 at the time of this study is 5%(50 cases/1000 citizens).

Now, let’s add antibody testing to the mix. Our Dr Deborah Birx expects antibody testing to be about 90% specific and sensitive to this SARS-CoV-2 virus: 10% of positives would be false positives (“90% specificity”); hence, these people do not have antibodies to the virus despite a positive test. Likewise, 10% of people actually with antibodies to the disease will be told they are negative (“90% sensitivity”). If we were to test everyone in Covania in a couple weeks (to allow antibodies to form), the test would produce results like this:

950 Covanians with no COVID-19 X 0.1 false positive rate = 95 false positives

50 Covanians with COVID-19 X 0.9 true positive rate = 45 true positives and 5 false negatives

Uh-Oh!

2/3rds of our “positive” results would be inaccurate. 140 Covanians might decide they can go out in the world without fear of contracting or spreading COVID-19, but 95 of those 140 would have FALSE reassurance!

Switch gears to a real world example: the Big Island of Hawaii. Prior to last week’s outbreak with 30 cases related to McDonald’s employees, there were 31 known cases. The positive test rate is app. 2.5%; well under 1% of the population has been tested, and the vast majority of tests have been administered to people with symptoms concerning for COVID-19, and often contact with people with known disease or travel to high-prevalence areas, given the difficulties with access to testing. It is safe to assume that the 97.5% of people tested as negative represent a substantially higher risk group than the general population at large. Moreover, the lack of any hospitalized patients would suggest that the 31 known prior cases are a fairly accurate count; even 31 cases would be expected to produce 2-3 hospitalizations in an average population, so the probability that many more than 31 cases existed a week ago, which would have allowed time to develop symptoms severe enough to warrant hospitalization, is quite low. An assumption of 50 true cases (prior to the McDonald’s outbreak) would be generous. This would indicate a prevalence of 50/200,000, or 0.025%. This is a fairly solid number – and a remarkably low one, at that – which we now must revise upwards given this new outbreak. For the sake of the model, let’s be neither optimistic nor terribly pessimistic, and say these 29 new cases mushroom into 150, bringing our total number of cases up to 200; at 200/200,000, we have a new prevalence of 0.1%.

Now let’s see how useful antibody testing would be:

199, 800 Hawaii County residents with no COVID-19 X0.1 false positive rate = 19,980 false positives

200 Hawaii County residents with COVID-19 X0.9 true positive rate = 180 true positives and 20 false negatives

20,000 people with the wrong test result! Most importantly: nearly 10% of the entire population would be reassured that they are safe to go back to work, to care for their elderly parents, to see patients in their clinics, etc – AND THEY WOULD NOT BE SAFE!  Only 180 people would benefit from the entire island being given this test and most of them would already have already received a positive PCR (“antigen”) test, anyway. The average person would be 111X more likely to get a false result than true!

Why does the test fail us so spectacularly? Taking a fairly accurate test for any disease and applying it to a population with a low rate of  that disease will always lead to a preponderance of false positives – poorly as it does on the Big Island, it will perform better in Covania, and better still in Lombardy. Improving the false positive rate makes a huge difference, as well. Increase specificity from 90% to 99% (and some antibody tests will claim this) and the numbers improve markedly: a 99% specific test in Covania with its 5% prevalence yields 5 true positives for every false positive – now we are starting to talk about a test worth running (with some caution). However, that same 99% accurate test in our Hawaii County example with 0.1% prevalence: “only” 1998 false positives for our 180 true positives. You are still 11X more likely to get bad information than good information if you take this test. The Abbott testing getting a lot of attention in California this week claims 99.6% specificity, a rather bold claim and one that needs to be tested in the real world; even at that rate, the Big Island would have nearly 800 false positives to accompany its 200 true positives – a 4:1 rate of bogus results and false reassurance.

In summary: unless COVID-19 spreads much more broadly through the population of Hawaii County, or someone, somehow, develops a true 100% specificity test, you are far more likely to be given a test with a false positive result than an accurate result. Pass on taking an antibody test, unless either these numbers change dramatically; or you were at a very, very high risk of exposure to the disease.

by Buzz Hollander MD

“When do we get to leave our house again?” This is a question we can expect to hear more and more in the days and weeks to come, along with its distant cousin: “Did we overreact to the threat of COVID-19?” Most of the media focus these days continues to involve mortality rates, overwhelmed hospitals, PPE, and ventilators, but as places like New York City and New Orleans begin to turn their respective corners, the “opening up” of the American economy will begin to take center stage. The problem is that predicting what will happen next is marred by uncertainty – the uncertainty begotten by bad data.

UNCERTAINTY #1: The severity of the first wave. 

As the world watched COVID-19 sweep through Wuhan, and then Lombardy, the “expert opinion” was: “look out – this is what your city will look like in 9-14 days!” This turned out to be eerily true in New York City and a handful of other American cities. However, even some of those places for which early modeling depicted scenes of horror in hospitals and morgues, like Seattle and San Francisco, emerged with one tenth the per capita suffering as New York. Why the difference? Many theories have evolved: the role of subways and intense population density; possible differences in viral strains; the prevalence of multi-generational housing; variations in climate; demographic and genetic/ethnic trends; timing and quality of the lock-down response. However, all of these theories have obvious flaws as primary explanations for the differences between regions, and none of these theories can be supported with great data. While it looks increasingly likely that Sweden will win its gamble with minimal government intervention by suffering only a modest increase in deaths compared to its locked-down neighbors; and that New York should have shut down more decisively and earlier, akin to California; and that the UK will fall somewhere in between despite its deliberately slow response; what is also clear is that there was really no way for the leadership of those different places to know what the best course of action would be. The risk of Lombardy hovered over everyone, and some places (New York) have endured that risk becoming reality; and others (San Francisco) have not.

UNCERTAINTY #2: The severity of future waves while awaiting a vaccine or effective treatment.

Some models have predicted that with relaxation of restrictions, the next waves will mirror the first. This does not make a great deal of common sense. We might reasonably expect that those most vulnerable to developing symptoms (and serious illness) to COVID-19 would have been disproportionately represented in the first wave statistics. In other words, assuming some people have advantages in fighting off this viral infection based on genetics or overall more vigorous health, these people would be less likely to have gotten ill from a mild exposure to COVID-19. People with less vigorous immune defenses would be more likely to fall ill with the same exposure. Hence, each successive wave of exposures leads to a lower rate of illness than the prior on a “per-exposure” basis. Future waves should be at least a little buffered from serious disease compared to the initial, on that basis alone. More importantly, there will also be a completely different set of obstacles to transmission for the virus to face than it did as it entered the first wave. Workplaces doing fever tests. A lot of people wearing masks. Handshakes, hugs and kisses, and nose-to-nose discussions, greatly reduced. Sporting events, large funerals, and choral practices banned for a spell. Barring a viral mutation to the worse, which virtually no one is predicting, it is hard to make the case that a second wave will be anything but gentler than the first. Whether it will be 10% gentler or 300% gentler, however, remains anybody’s guess.

UNCERTAINTY #3: The role of herd immunity in reducing the severity of the pandemic.

Some respected voices in the debate of how best to manage this pandemic have taken the side that the virus is already more common than we know, and that “herd immunity” is quietly building, and therefore we should relax restrictions and let it finish its natural course.

Herd immunity – the concept that enough of the population has antibodies to a given infectious agent that its transmission becomes so low as to eliminate it as a problem – is a popular concept to invoke amongst those who number themselves in the “business as usual” crowd. This is the sunny daydream of epidemiologists studying COVID19 with an optimistic attitude. The idea would run like this: a large proportion of people in an affected population get mild or no symptoms, never get tested or suspected of exposure, and when we have accurate antibody testing we will find that the majority of people after the first wave of illness will have already been exposed and have immunity to COVID19. If this number breaks over 60% or so, future spread of COVID19 will be tempered by the lack of receptive “COVID19” potential victims, and a population could see a slow, manageable ascent to herd immunity, when 95+% of a population has protective antibodies to a virus. Sounds great, doesn’t it? The problem is the lack of data to back up these suppositions. In places where entire populations have been tested, i.e. Vo, Italy, the rate of infection tends to be low, well under 5%; and even heavily-exposed populations like the beleaguered Diamond Princess cruise ship was under 20%. Unless the rate of false negative antigen tests is very much higher than we believe, there is almost no way that more than 10 or 20% of even the hardest hit regions, like northern Italy, have antibodies to this illness. If that is the case – imagine how bad Lombardy, Madrid or New York City would have looked had the authorities there not enforced social distancing!  Conservatively, double or triple the case rate in New York, and run that through your imagination. Barring the deus ex machina scenarios that mild disease already abounds and has avoided testing, we develop a highly effective treatment, or the virus mutates to be less lethal, there are two ways to achieve herd immunity to COVID19: let a potentially huge number of people die, many times over what we are seeing with social distancing requirements; or wait for a vaccine.

UNCERTAINTY #4: The cost of a government-imposed shut-down vs the disease itself.

This is a hard one to even talk about. You hear headlines like, “One unnecessary death from COVID-19 is one death too many”; and the governor of Hawaii expressing condolences to the family of each victim of COVID-19. The hypocrisy is obvious. No such announcements are made during flu season, despite the fact that Hawaii is likely to see only a fraction of its residents die of COVID-19 than the flu this year. Tobacco is still legal in the United States despite its excess mortality being far greater than any viral infection. No one is padding every tree on every roadway in America with thick cushions to reduce the incidence of traffic fatalities despite their causing 35,000 deaths annually in the US. Why not? Almost any intervention to reduce mortality comes at a cost; a cost to liberty, or a financial cost. The means to reduce the impact of COVID-19 have both these types of cost.

From a public health perspective, the ideal approach to re-introducing commerce would involve a lot of government restrictions: restricted travel from high-prevalence regions; limited socializing for the elderly; bans on large gatherings; possible continued school closures; and perhaps even requirements to show proof of prior infection, or of lacking high-risk co-morbitidies before allowances to travel freely throughout society. This would have an immediate cost to liberty; and perhaps a long-term price even higher, as the populace gets comfortable with being monitored in ways that seemed Orwellian a few months ago.

Then there is the economic cost. This has been well-documented. The deliberate shut-down of most sectors of the national economy has already had a disturbing effect on the unemployment rate; is triggering a recession, possibly a long and profound one; and has led to a massive government bail-out, one which will either lead to increased taxes, a devalued dollar, long-term limited growth, and/or all of the above. Such economic costs also come at a price to human health. People lose their health insurance. Technologies that can save lives get put on hold. Suicide rates climb. Nutrition gets compromised. Although in the short term, mortality in the western world tends to improve in a recession because people do fewer higher-risk activities, we lack numbers on what happens in the long term. Was there an alternative to government shut-downs?

 Letting COVID-19 “run its course” would have its own cost, too. Not just with a higher death rate, either. These are all very much “back-of-napkin” projections, but it seems reasonable to think the cases and deaths could easily have been many times what we have actually seen with strict lock-down responses. If you double the number of hypothetical New York ICU beds to, say, an additional 30,000, and calculate the expense of an average ICU stay to be $50,000 (a conservative estimate, to be sure), some astronomical numbers begin to appear: $1.5 BILLION in New York alone from additional hospital expenses. Then there is the loss to the work force of more people getting ill, and more people caring for ailing family members. Add in a paranoid populace not going out to eat  much or showing up for work of their accord, taking its own steep toll on the economy. Would a lot of tourists have showed up at Times Square during a hypothetical COVID-19 outbreak even worse than we have witnessed? You can make the case that an expensive recession was inevitable in those places getting badly hit by COVID-19, even without government-enforced lock-downs; and that the “savings” in terms of minimizing deaths and severe illnesses are well worth any additional cost. A contrarian, however, could make the case that on a national level the risk of “having a few New Yorks” being even worse than they were would have been a fair trade for not having the entire country suffer economically the way it has.

One thing is certain: we will never know with certainty who would be “right.” As a country, though, there are some hard decisions to make in the weeks and months to come, and they will have to made with bad data shedding its murky light. There is a growing chorus of how the country can be “re-opened”, however, and there is good reason for hope that this can be done without a repeat of the damage seen from COVID-19’s first wave. This is a certain topic for a future post!