SARS-CoV-2, the virus responsible for the COVID-19 pandemic, is a new/novel virus for the human race. One of the pillars of managing this virus and the pandemic is accurate testing. Testing for the SARS-CoV-2 virus remains a challenge with regards to the accuracy of tests on the market. Epidemiologically, any diagnostic or laboratory test is always evaluated on two key metrics: it’s specificity and sensitivity rate.
Specificity is the likelihood that the COVID test results are truly positive. That is, could a similar Non-COVID-19 Coronavirus be responsible for the positive result? A COVID test with a high specificity (i.e., reliable) will have a LOW false positive rate. Generally speaking it is desirable to have a test with high specificity. However, high specificity is attained at the expense of sensitivity. It is desirable to have a test with high specificity and sensitivity, but almost impossible to create a perfect test.
SARS-CoV-2 in its worldwide prevalence in the general population is still low. Even with the current pandemic raging, the prevalence and incidence is still low based on identified active cases (Some 5 million active cases identified worldwide out of a population of about 7.7 billion as of May 21, 2020). Aside from the diagnosed positive cases, it is estimated that the prevalence of coronavirus infection is more around 5% in the general population. We commonly associate a low prevalence of disease with increased false positive rates when testing. The current low prevalence of COVID infection affects the accuracy of SARS-CoV-2 tests coming into the market for use, with the risk of false positives becoming a major problem. Test manufacturers and regulators alike will have to be diligent and guard against a high number of false positives when developing and approving tests.
Sensitivity of a SARS-CoV-2 molecular test or antibody test is the likelihood of how accurately the test will detect the virus or presence of the different types of antibodies. That is if you are infected with the virus, will the test correctly identify that you are infected, or will it not identify the virus in your respiratory system, resulting in a false negative. A highly sensitive test (i.e., accurate) will have a LOW false negative rate.
Most of the tests granted emergency use approval by the FDA during this pandemic have limited data with regards to their reliability and accuracy. Scientists at the Cleveland Clinic found that one popular rapid genetic test (the widely promoted Abbot test) told users they didn’t have the virus when they actually did (false negatives) about 15 percent of the time (1). Meanwhile, researchers at UCSF California studied 14 antibody tests on the market, some that were granted emergency use authorizations by the Food and Drug Administration (FDA), and found that only three of these tests delivered consistent results (1). Given the 5% prevalence rate, if a serological test has about 90% specificity, its positive predictive value (PPV) will be 32.1% – meaning nearly 70% of positive results will most likely be false. At this same disease prevalence, a test with 95% specificity will lead to a 50% chance that a positive result is wrong. Only at 99% specificity does the false positive rate become anywhere near acceptable, and even here the chances are that 16% of positive results would be wrong.
We know that a number of individuals with COVID infection are asymptomatic, sometimes for weeks, increasing the transmissibility of the disease in the population. For such cases it is important to have tests that can look for active infections with a high degree of sensitivity and specificity (low number of false negatives and false positives). If we can quickly identify such cases and isolate them, then we can prevent the rapid spread of this contagion.
The table below summarises test accuracy data, as of May 06, 2020, from selected manufacturers for their serological Covid-19 tests. However, when reviewing this data it has to be stressed that the validation tests these companies have performed have varied widely in size; Abbott’s antibody assay was tested on 1,200 specimens, whereas Epitope’s tests were run on only 54 samples from healthy people, and just 20 and 30 cases of PCR-confirmed Covid-19 cases for the IgM and IgG tests respectively.
Accuracy & Reliability of Selected COVID 19 Tests in the US & EU Markets:
|Company||Testing Device Name||US Status||EU Status||Sensitivity (%)||Specificity (%)|
|Abbott Laboratories||Abbott Sars-CoV-2 IgG Test||Marketed||Filed||100||99.5|
|Becton Dickinson/ Biomedomics*||BioMedomics Covid-19 IgM/IgG Rapid Test||Marketed||Marketed||88.7||90.6|
|Cellex*||Cellex qSars-CoV-2 IgG/IgM Cassette Rapid Test||Marketed||Unclassified||93.8||95.6|
|Creative Diagnostics||Creative Diagnostics Sars-CoV-2 Antibody Elisa||Marketed||Marketed||94.5||100|
|CTK Biotech||OnSite Covid-19 IgG/IgM Rapid Test||Marketed||Unclassified||96.9||99.4|
|Epitope Diagnostics||EDI Novel Coronavirus Covid-19 IgG Elisa Kit||Marketed||Unclassified||100||100|
|Epitope Diagnostics||EDI Novel Coronavirus Covid-19 IgM Elisa Kit||Marketed||Unclassified||45||100|
|Intec Products||InTec Rapid Sars-CoV-2 Antibody (IgM/IgG)||Marketed||Marketed||95.2||98|
|Nirmidas Biotech||Nirmidas Biotech Covid-19 (Sars-CoV-2) IgM/IgG Antibody Detection Kit||Marketed||Unclassified||93.8||99.5|
|Ortho-Clinical Diagnostics*||Vitros Immunodiagnostic Product Anti-Sars-CoV-2 Total Reagent Pack||Marketed||Unclassified||83.3||100|
|SD Biosensor||Standard Q Covid-19 IgM/IgG Duo Test||Marketed||Unclassified||81.8||96.6|
|Note: All accuracy claims made by the companies. *Tests with FDA emergency use authorisation. Source: EvaluateMedTech & company websites.|
Similar to most processes associated with COVID 19, since the emergence of the infection, tests for active infection and/or for antibodies are evolving daily by different clinical research laboratories, academic health organizations, and biomedical companies worldwide. However, in the US we are still facing a challenge with regards to ready and widespread availability of accurate and reliable COVID tests for both the molecular PCR test and the serological tests for the different antibodies. The situation is both dire and murky to say the least.
At this point, it is well recognized that both the molecular tests for Covid-19 (the nasal PCR that look for active infections) and the serological antibody tests (IgM for more recent and IgG for past infections) have had significant issues with respect to their accuracy. Shortage of good tests in turn means that we will not know how many active cases are out there in the public, or how many have recovered with a degree of short or long term immunity conferred on them to decrease the risk of contraction and/or rate of reinfection.
2. Data compiled by EvaluateMedTech.
3. Loeffelholz MJ, Tang YW. Laboratory diagnosis of emerging human coronavirus infections – the state of the art. Emerg Microbes Infect. 2020;9(1):747–756. doi:10.1080/22221751.2020.1745095
4. Sheridan C. Fast, portable tests come online to curb coronavirus pandemic. Nat Biotechnol2020. [Epub ahead of print] doi:10.1038/d41587-020-00010-2 pmid:32203294