One Hundred Eleven Good Reasons to Avoid Antibody Testing for COVID-19

by Buzz Hollander MD

Antibody testing for COVID-19 is the new darling of the media. It seems that everyone wants to know whether that little scratch in their throat they had in early February might just mean they have antibodies to SARS-CoV-2! As scientists are interviewed touting the impressive accuracy of these tests, interest continues to grow about antibody testing being the key to “re-opening the economy” as people are able to learn their immune status. But is this going to be as good as it sounds?

Let’s look at two examples to better understand; one hypothetical, the other real world:

Covania is a small town of 1000 people in the outskirts of a metropolitan area with a significant COVID-19 outbreak. 5 Covanians ended up needing to be hospitalized in the initial wave of the outbreak, two required a ventilator, and one died of COVID-19. Officially, 25 Covanians were tested and 10 were found to be positive, including the 5 hospitalized patients. However, in our hypothetical example, 50 Covanians actually had COVID-19, although 10 never exhibited symptoms. The very high test positive rate of 40%, and the rather high testing rate of 2.5% of the population, mirror the numbers for New York City rather well; as does the deaths per million (DPM) of 1000 DPM that Covania’s single death produces. Covania’s true prevalence of COVID-19 at the time of this study is 5%(50 cases/1000 citizens).

Now, let’s add antibody testing to the mix. Our Dr Deborah Birx expects antibody testing to be about 90% specific and sensitive to this SARS-CoV-2 virus: 10% of positives would be false positives (“90% specificity”); hence, these people do not have antibodies to the virus despite a positive test. Likewise, 10% of people actually with antibodies to the disease will be told they are negative (“90% sensitivity”). If we were to test everyone in Covania in a couple weeks (to allow antibodies to form), the test would produce results like this:

950 Covanians with no COVID-19 X 0.1 false positive rate = 95 false positives

50 Covanians with COVID-19 X 0.9 true positive rate = 45 true positives and 5 false negatives

Uh-Oh!

2/3rds of our “positive” results would be inaccurate. 140 Covanians might decide they can go out in the world without fear of contracting or spreading COVID-19, but 95 of those 140 would have FALSE reassurance!

Switch gears to a real world example: the Big Island of Hawaii. Prior to last week’s outbreak with 30 cases related to McDonald’s employees, there were 31 known cases. The positive test rate is app. 2.5%; well under 1% of the population has been tested, and the vast majority of tests have been administered to people with symptoms concerning for COVID-19, and often contact with people with known disease or travel to high-prevalence areas, given the difficulties with access to testing. It is safe to assume that the 97.5% of people tested as negative represent a substantially higher risk group than the general population at large. Moreover, the lack of any hospitalized patients would suggest that the 31 known prior cases are a fairly accurate count; even 31 cases would be expected to produce 2-3 hospitalizations in an average population, so the probability that many more than 31 cases existed a week ago, which would have allowed time to develop symptoms severe enough to warrant hospitalization, is quite low. An assumption of 50 true cases (prior to the McDonald’s outbreak) would be generous. This would indicate a prevalence of 50/200,000, or 0.025%. This is a fairly solid number – and a remarkably low one, at that – which we now must revise upwards given this new outbreak. For the sake of the model, let’s be neither optimistic nor terribly pessimistic, and say these 29 new cases mushroom into 150, bringing our total number of cases up to 200; at 200/200,000, we have a new prevalence of 0.1%.

Now let’s see how useful antibody testing would be:

199, 800 Hawaii County residents with no COVID-19 X0.1 false positive rate = 19,980 false positives

200 Hawaii County residents with COVID-19 X0.9 true positive rate = 180 true positives and 20 false negatives

20,000 people with the wrong test result! Most importantly: nearly 10% of the entire population would be reassured that they are safe to go back to work, to care for their elderly parents, to see patients in their clinics, etc – AND THEY WOULD NOT BE SAFE!  Only 180 people would benefit from the entire island being given this test and most of them would already have already received a positive PCR (“antigen”) test, anyway. The average person would be 111X more likely to get a false result than true!

Why does the test fail us so spectacularly? Taking a fairly accurate test for any disease and applying it to a population with a low rate of  that disease will always lead to a preponderance of false positives – poorly as it does on the Big Island, it will perform better in Covania, and better still in Lombardy. Improving the false positive rate makes a huge difference, as well. Increase specificity from 90% to 99% (and some antibody tests will claim this) and the numbers improve markedly: a 99% specific test in Covania with its 5% prevalence yields 5 true positives for every false positive – now we are starting to talk about a test worth running (with some caution). However, that same 99% accurate test in our Hawaii County example with 0.1% prevalence: “only” 1998 false positives for our 180 true positives. You are still 11X more likely to get bad information than good information if you take this test. The Abbott testing getting a lot of attention in California this week claims 99.6% specificity, a rather bold claim and one that needs to be tested in the real world; even at that rate, the Big Island would have nearly 800 false positives to accompany its 200 true positives – a 4:1 rate of bogus results and false reassurance.

In summary: unless COVID-19 spreads much more broadly through the population of Hawaii County, or someone, somehow, develops a true 100% specificity test, you are far more likely to be given a test with a false positive result than an accurate result. Pass on taking an antibody test, unless either these numbers change dramatically; or you were at a very, very high risk of exposure to the disease.