Mass screening: is this the answer?

In the news again this week is the message that new approaches to testing for Covid-19, particularly using self-administered saliva tests, with just a 15 minute wait for a result, could be the answer to the pandemic.  Indeed, there are many examples across the world where everyone in a region or organisation (such as a workplace or university) has been so tested.  In a previous post on this blog, I mentioned that such a saliva test was just ‘around the corner’.  There are 7 important questions that now need to be addressed when considering the value from such a strategy:

1. Tests for screening or diagnosis? 
2. What is screening?
3. Do screening tests for Covid-19 give the right answer?
4. What are the causes and consequences of false negative tests?
5. What are the causes and consequences of false positive tests? 
6. Do we know what the false negative and false positive rates are for the various Covid-19 tests?
7. Are there any other aspects of the test to consider?

1. Tests for screening or diagnosis?

• Early in the pandemic, tests were developed based on mouth and nose swabs to determine if individuals with typical symptoms of Covid-19 actually had the virus
• The test was therefore used to diagnose an individual as ‘a case’
• The positive rate of tests in those with symptoms has varied depending on what symptoms lead to the test and the prevalence of other diseases such as seasonal flu at the time of the test
• In the early stages of the pandemic, when tests were not widely available, proving the diagnosis in people who were mildly unwell was considered unnecessary as it would not change the advice (stay at home etc)
• Even though there was still no effective treatment for mild cases, the need for a diagnostic test later became more relevant in a negative sense: ie showing someone was negative allowed that person to resume normal activities
• Over time it has become apparent from testing random samples of the population that a significant proportion of ‘cases’ of Covid-19 do not have symptoms at the time of a test and might never develop them

• In these data from the UK national survey, currently only about a third of those who tested positive had symptoms.  Indeed the prevalence was much lower in the summer, the symptomatic rate was under 20%
• Having asymptomatic infection may not be a problem for the individual but it is for greater society, given that such individuals can pass the infection on to others
• Thus, the question is raised as to the value of Covid-19 testing in  asymptomatic people for the purposes of screening

2. What is screening?

• In a strict epidemiological sense, screening is defined as the application of a test to identify cases that had not yet, or might never, become apparent
• Screening of individuals is of value only if there is benefit of earlier detection
• For example, it is no use introducing a cancer screening programme if the people who are detected gain no benefit from the earlier detection 
• Thus, screening of individuals for a disease is only of value if there is something that can be done to change the outcome, for example earlier treatment.
• By the same token screening of populations is only of benefit if that leads to an intervention that would reduce the population impact of the disease
• Thus, screening asymptomatic people in the general population for Covid-19 is only useful if there is something that would be done that would
  • Reduce the spread of the infection
  • Allow sections of society to function normally 
• The testing on its own is useless unless there are effective measures that can achieve those goals
• For the rest of this post, I will assume that the test results will lead to successful actions in terms of contact tracing, quarantine etc that will bring about the desired reduction in the rates of infection in the target populations 
• The rest of this post thus considers how important that the screening test will give the right answer enough of the time

3. Do screening tests for Covid_19 give the right answer?

• Although this appears a simple question, it is really challenging to answer but could be crucial in determining whether there is any benefit from mass screening
• No test for Covid-19 is completely accurate all the time!
• There are several reasons why you may get the wrong result

• Looking at the diagram above we can see that there after testing, people will fall into one of four groups:
  • Have the infection and the test was positive (the green figure)
  • Have the infection but the test was negative (the black figure)
  • Don’t have the infection but the test was positive (the red figure)
  • Don’t have the infection and the test was negative (the blue figure)

At the end of the blog for those who are interested I show how we calculate the rates of false negatives and false positives

4. What are the causes and consequences of false negative tests?

• These are some of the reasons why there might be false negatives from a Covid test:
  • The person actually was infected but was too early to show up in any test
  • The person was infected and was not producing enough material to show up in that test
  • The person was actually infected but there was a problem with the test itself:
    •  A swab test did not collect samples from the correct areas of the mouth and nose
    • A saliva test sample was not collected sufficiently well  
    • Something went wrong with the  storage and processing
    • The test in the lab gave a false reading
    • The threshold for saying a test was positive was too high (this will be discussed further below)
• There are a number of consequences of the false negative rate in terms of screening, both for the individual and society

5. What are the causes and consequences of false positive tests?

• There are some reasons why there might be false positive from a Covid test:
  • Contamination of a swab sample by Covid-19 in the lab, on the glove of the handler etc
  • Contamination by Covid-19 of the chemicals and other materials used to do the test
  • Test is positive but due to the presence of a different but harmless corona virus
  • Test is positive but due to the presence of genetic material that was not from Covid-19 

• There are a number of consequences of the false positive rate in terms of screening both for the individual and society

6. Do we know what the false negative and false positive rates are for the various Covid-19 tests?

• This is a challenging question as to know when a test is giving a wrong result; it implies that there is a way we can find out what the true result was, other than from the test!
• In the lab,  a new test is analysed on samples from patients with known virus loads from more stringent tests and who may have obvious clinical infection with Covid-19 – but that might not be so useful in the real world as we want a new test to pick up infections in people with smaller viral loads who may, for example, be asymptomatic
• Even in the lab with the ideal conditions, both false positive and false negative rates for Covid-19 tests are around 5%
• We really do not know what these rates are in community -wide settings
• A paper in the Lancet at the end of September suggested a false negative rate could be as high as 30%
• Researchers and test developers have compared new saliva tests with  swab tests as in the example below.  But that study can only give the rates of positives and negatives relative to each other

• Some research has estimated the rate of false negatives by doing repeat tests after a few days, to see if people are positive the second time
• This may be sensible but of course the person’s infection status might have actually changed in that time
• So there can only be a best guess as to what are the false negative and false positive rates for any new test

7. Are there any other aspects of the test to consider?

• One final comment is that in all the lab tests the result is not a clear cut yes/no and a decision is made as to what reading would be considered as positive
• If that threshold is made much less stringent -‘raise the bar to being positive’, then this would reduce the number of false negatives – but of course raise the number of false positives
• If that threshold is made much more stringent – ‘raise the bar to being positive’ then this would increase the number of false negatives but of course reduce the number of false positives
• It is a difficult decision where to ‘put the bar’ and this will depend on the relative harmful consequences of a being a false positive or a false negative

Conclusions:

• Any speedier test that can be applied to populations and that can lead to effective action to reduce transmission of infection is of course welcome
• Testing alone which does not change behaviour or policy is a waste of time
• My aim in this post was to emphasise that we should accept there is no perfect test but we do need to understand the imperfections of the tests that are coming into use

Appendix: Calculation of false negative and false positive rates

• (In the diagrams below I have used the same colouring as in the image above of the different results)
• For any test we can calculate the percentage of all results that should be positive but were wrongly called negative: this is called the False Negative rate* and is calculated as below (multiplied by 100 to give a percentage):

• This seems to be counterintuitive, as you would think the false negative rate should be the proportion of all negative tests that are false.
• But the aim is to make sure that as many as possible of the true cases are identified
• Similarly for any test we can calculate the proportion of all tests that should be  negative but were wrongly called positive: this is called the False Positive rate* and is calculated as below (multiplied by 100 to give  percentage)

• That also seems to be counterintuitive, as you would think the false positive rate should be the proportion of all positive tests that are false.
• But the aim is to make sure that as many as possible of those who are not cases cases are also correctly identified

If you would like to receive email notifications of new posts on this blog just click this link and enter your email at the bottom of the home page: https://makingsenseofcovid19withs.com