Voluntary Testing vs. Random Testing

Way back at the beginning of the pandemic epidemiologists were urging us to establish random testing.  When you have voluntary public testing like we do now, people make their own decisions to opt-in for a test.  In social science this is referred to as a self-selected sample.

The problem with that approach is self-selection bias.  This bias means factors other than the thing you are trying to measure influence the test results.  Worse, these factors are unknown and can vary from respondent to respondent.  

A good non-pandemic example of this is on-air issue polls run by media outlets.  A poll asking whether the respondent supports gun rights is going to turn out very differently when run on MSNBC than when run on Fox News.  Neither is going to give an accurate picture of support in the country at large.

For voluntary COVID-19 testing, the bias is due to reasons people choose to be tested.  Maybe they are hypochondriacs.  Or maybe they want to be sure they're not actively infected before visiting someone (current rules in Arizona require people to be tested before visiting loved ones in residential care facilities).

But probably the main reason people voluntarily seek a test is that they think they are infected or have been exposed to someone who is.  This means tests in our current pool are biased toward people who have symptoms or have had a dangerous exposure.  Accordingly, the positive test rate for these tests will be inflated, perhaps by a lot.  

The solution to self-selection bias is random testing.  Instead of (or in addition to) allowing people to self-select for tests, we randomly choose people for testing.  We do this in large enough numbers that the resulting positive test rate is statistically significant—meaning that results from a sample of that size will give the accurate positive test rates in the population 95% of the time (to oversimplify slightly).  

It turns out that random sampling makes a huge difference.  Arizona's three major state universities randomly select their students and employees for testing in numbers that create statistical significance.  ASU's numbers can be found here.  They show a 2.9% positive test rate (averaged over on-campus students, off-campus students, and staff) for the week ending January 2nd.  For the same period the state overall was reporting 36% positive tests according to the Biodesign dashboard (Maricopa county numbers would be a better comparison, but the dashboard only offers a county-breakdown for yesterday's testing numbers).

The situation is much the same for the University of Arizona.  Their numbers can be found here.  They report an overall 3.4% positive rate for the ten days ending January 6th, versus a seven-day average of 38.49% statewide on that date.  NAU's most recent data is from November and shows a 2.9% positive rate.  They helpfully give the rate for the same period in Coconino County, which was 9%.  

So the self-selected samples have 3x to 12x higher positive test rates than random sampling.  That shows that even at relatively low infection rates our hospitals are filling up and a lot of people are dying.  If rates were really what the voluntary testing shows, it would be an Apocalypse.  

There are many other reasons why random testing is a good idea.  They are reviewed in this article from Slate in April 2020.  But a big one is that random sampling would reveal asymptomatic cases, which scientists think is the primary way the virus is spread.

I would be remiss if I didn't conclude by pointing out how ignorant President Trump was being when he repeatedly claimed over the spring and summer that more testing means a worse case rate, making him look bad.  In fact, the exact opposite is true. Why? Because most people (about 97% according to the above numbers) are not infected with the virus.  The more of them you get into the testing pool, the more negative tests you will have compared to positive tests, so the lower the positive test percent will be.  You'd think a genius would be able to understand that.

You'd also think the greatest country in the world with the greatest health care system in the world would have established widespread random testing 10 months ago.  We could have, but sadly we didn't.

Update 1/21

I spoke to an epidemiologist who pointed out that university populations aren't a great stand-in for the populations of the counties they're in. Students have better access to medical care than their counterparts in the general population, and faculty/staff are better able to work remotely and limit their exposure to the virus.  There no equivalents of grocery store clerks or factory workers, etc., in a university.  So the university random testing programs almost certainly underestimate the true positive test rate in the population.