‘Random’ Thoughts About the Randomized Testing Process
By Dr. Christopher Alterio
As we enter the third month of randomized COVID-19 testing – a key part of Keuka College's Spring 2021 Reopening Plan – we've heard questions about the process. Students who have been selected for repeated testing might point to a classmate who hasn't been selected at all and ask how a randomized system can seem less than random. It's a good question. And there are good answers. Read on!
Randomization is a statistical method that ensures an equal probability of being assigned to a group – and that’s a desirable method when trying to apply an unbiased system to a total population.
Sometimes when you are looking at a problem you might want more than simple randomization. COVID-19 surveillance is a good example, especially when you want to have a constant “eye” on subgroups within a population – you want to know early if Floor X of Residence Hall Y is having a particular problem with spread. After all, that’s what we all experienced during our campus outbreak in the Fall of 2020: some floors, residence halls, and subgroups had more community spread than others. It’s too late to learn and understand those details when the spread is already beyond a point of control.
That’s exactly why we moved to a population-level surveillance method.
There are different ways to design a surveillance system to make sure that there are “eyes on” all different aspects of a population. You can stratify your sampling so that you are randomly selecting people from each floor. Or you can maintain simple randomization and have a background “whole dorm” surveillance by monitoring viral load in wastewater. There are many different design options.
When we were planning our campus surveillance model we opted for stratified sampling because we thought it was the most direct method to be sure we were monitoring every subunit within each residence hall. Over the course of the semester, we have gotten some feedback that the randomization doesn’t “feel” so random to some students who have been selected several times.
How could that happen, if we are selecting students randomly?
Well, we checked our random selection algorithms, and they are indeed random. That was not the problem. Then we began to look at other factors that could impact randomization and why our stratified method was running into challenges. It’s true that stratification methodologies can be problematic when you can’t cleanly separate people into groups. Is that what happened – we wondered?
After some analysis, we began to understand that when we selected the stratified randomization method that we did not know if athletics would be happening, so we didn’t consider the impact of that sub-population into the sampling methodology. Then, when it turned out that athletes were having their seasons in a modified way, we pulled them from the general surveillance pool because they were undergoing their own separate surveillance – sometimes multiple tests per week – in accordance with NCAA guidelines.
What we didn’t understand at that decision point was that the distribution of athletes in the residence halls was not equal. In fact, athletes are often, but not always, clustered in the residence halls in pockets. That means that Floor X of Residence Hall Y might have an over-representation of athletes. If they are pulled from the general surveillance pool, the remaining students in that stratified category start to shrink to the point that they might be “over-selected,” even though the algorithm is still random.
It is a fair criticism to wonder why this wasn’t anticipated, but we really didn’t know that athletics were going to take place. We also don’t routinely design population-level surveillance testing in response to a pandemic!
This is a great example, though, of how Keuka College is a place of lifelong learning – and just as would happen in a designed research study, if you find a design flaw you have to talk about it. Now, since surveillance testing is not a research study, we are not obligated to maintain the exact same design to maintain fidelity to a process. So we are fixing the design.
Since we are not seeing significant challenges with campus transmissibility, we feel comfortable going to a simple randomization at a higher percentage. If an athlete is randomly selected for campus surveillance testing and they are already engaged in the separate athletic surveillance, we will count the “other” test as fulfilling our campus surveillance. We will be bumping up our sample size to make sure we are working toward an equal chance to test non-athletes. Is this perfect, either? Probably not! This is the challenge of designing a real-world program with individuals who have different testing requirements and who don’t always neatly fit into distinct categories.
To substitute for lack of stratification by floor and residence hall, we might increase our testing of a sub-population, but that will only happen if we begin to see a problem with transmissibility. What that means is that if we see suspicious patterns of transmission within a group, we might selectively increase our surveillance of that population for a period of time.
The bottom line on all of this is that we are engaged in this testing process to help keep the campus as free from COVID-19 as possible. So far this semester, you all have been very successful with that! We also know that many people are choosing to get vaccinated, and that’s helpful and we encourage everyone to consider that. We also know that some people have had COVID-19 and now carry some immunity. That immunity is great, too.
In addition, we know that there is a huge campus commitment to mask-wearing, social distancing, washing hands – all the basics – and we really appreciate all of those efforts. All of these things combined are helping to keep us on campus this semester.
So talk about this issue in statistics and research methods and population health classes. This is an amazing time to really apply those ideas to real-world situations. Also, keep asking questions and providing feedback, because it is the whole-campus effort that allows us to Believe in What We Can Do Together!
Associate Professor of Occupational Therapy Dr. Christopher Alterio is Keuka College's Division Chair of Applied Health and Wellness, Director of Disability Services, and Chair of the College’s Reopening Taskforce.