SARS-Cov-2 PCR False Pos

[wholeheartedly]

For background- former lab tech at big tertiary place, current infection preventionist in rural shop and home recovering from surgery + sick so really exhausted.

Could use input on this situation:

The state health department did facility wide covid-19 PCR testing on our nursing home residents and staff. One resident tested positive, no one else. The resident is asymptomatic. Since none of the staff tested positive, one of our physicians is adamant it was mostly likely a false positive and wanted to do a repeat test.

My take was that while false pos are always possible, the specificity of the covid PCR tests is pretty good and false pos are much less common and false negatives are a much more common/greater concern, so if we did test the resident again and it came back negative it would be more likely the second test was a false negative than that the first was a false positive.

He stated that he’d seen a high false positive rate in PCR noted from groups like IDSA, AAFP, JAMA, and others I can’t remember. He noted upwards of a 20% false positive rate.

That’s not consistent with what I’ve seen in everything I’ve read about the performance of the PCR tests and seems more in line with the false neg rate or concerns about the false pos issue in antibody tests. But I’ve been out for surgery, so perhaps I’ve missed more recent data that’s come out?

Can you guys help me out whether you’ve seen that kind of false pos rate noted for any of the common US PCR tests? I’m in pretty miserable condition right now and not up for sifting through all the lit again, but worried they’re going to retest and try pull her out of iso early.

Thanks

---

Members don't see this ad.

 

[caffeinegirl]

See False positives in reverse transcription PCR testing for SARS-CoV-2
False positives can occur especially when prevalence is low

 

[Sulfinator]

If the person in question were to also test positive for IgG anti-SARS-CoV-2 antibodies (either now or in the next couple of weeks), then I would think that would make it more likely that the PCR test was a true positive. Still, the point remains that false positives (for any test) are real things that one has to consider.


Sent from my iPhone using SDN mobile

 

[wholeheartedly]

Of course false positives need to be considered with any test. The question was has anyone seen data in line with what he suggested in anything you’ve read? only 70-80% specificity for PCR in IDSA, NEJM, JAMA, AAFP? I couldn’t find anything like that and want to make sure I’m not missing something important. Everything I’ve read suggest the PCR tests are quite specific.

 

[Napoleon1801]

Of course false positives need to be considered with any test. The question was has anyone seen data in line with what he suggested in anything you’ve read? only 70-80% specificity for PCR in IDSA, NEJM, JAMA, AAFP? I couldn’t find anything like that and want to make sure I’m not missing something important. Everything I’ve read suggest the PCR tests are quite specific.

It certainly depends on the method, but most PCR tests I've come across have 99% specificity and are designed not cross react with the four seasonal coronaviruses (or other respiratory pathogens). A 20% false positive rate would be unheard of. He needs to provide some sources to back that claim up.

The real concern with COVID PCR testing is the sensitivity of the test. That may be where this "70-80%" number is coming from. However, a test with decreased sensitivity (but high specificity) would lead to more false negatives. If you think this patient has a false positive result, a second test be a different method, or even an antibody test would clear things up. Asymptomatic carriers of this virus are common at this point.

 

[Allegri]

You are attempting to interpret a single positive test. The number you should care about is the positive predictive value, or the proportion of positive tests that are true. This is related to specificity but not the same thing, because it is dependent on disease prevalence. Low disease prevalence leads to a higher proportion of positives being false. We don't know exactly what the prevalence of covid is. But if you are in a lower prevalence area, then I think the possibility that that single positive test is false, among all those negatives at the same place, is actually pretty good. I think a re-test is reasonable.

 

[wholeheartedly]

You are attempting to interpret a single positive test. The number you should care about is the positive predictive value, or the proportion of positive tests that are true. This is related to specificity but not the same thing, because it is dependent on disease prevalence. Low disease prevalence leads to a higher proportion of positives being false. We don't know exactly what the prevalence of covid is. But if you are in a lower prevalence area, then I think the possibility that that single positive test is false, among all those negatives at the same place, is actually pretty good. I think a re-test is reasonable.

Thanks.

I’m sorry for the confusion, I gave the scenario for this patient for background, my specific question was regarding if anyone has seen any data supporting his numbers because I’m too sick to sift through everything right now and check. But I’ve seen nothing so far remotely like those numbers anywhere, so I’m quite confused.

this resident is in a nursing home and the state health dept has a protocol for how long she’ll stay in isolation, we have no other covids there so risk to her from a false pos staying in iso is very low compared to risk of mistakenly moving her back to the general population early.


I’d just like to make sure I’ve got the most accurate data moving forward. Otherwise, yes, PPV would be a critical piece of the picture.

 

[Med Director New England]

Why is the person convinced the positive is a FP? Much more likely that some of the negatives in the group tested are FNs.

 

[gbwillner]

This is why a thorough validation is necessary.

 

[SetecAstronomy]

Don't forget about pre-analytics. If they didn't jam the swab far enough into the nose, you can get false negatives. Likewise, there are rumors (maybe published?) about negative nasal swabs, but positive results in lung samples. The examples are obviously about falso negatives, but the concept is still valid. Contamination at any step can yield a false positive.

Then again, there are definitely reports of asymptomatic carriers.

 

[wholeheartedly]

Why is the person convinced the positive is a FP? Much more likely that some of the negatives in the group tested are FNs.

That was my thought and what I said in our incident command mtg.

Might have something to do with the fact his gf is the new director of nursing at the nursing home and it’s a pain in her ass to have this resident on precautions. Anything she doesn’t like, gets me (and our CEO, soon to be CEO, the chief of staff, the practice mgr, and my boss) a nasty email from him. Example, having her febrile etc. employees stay home from work following health dept. return to work guidelines.

He’s reporting seeing this 70-80% specificity rate in places like JAMA, NEJM, IDSA, and AAFP. I thought I’d done a pretty good job keeping up with the lit and everything I’ve seen suggests pretty good specificity and not the greatest sensitivity. But I’m post op and feeling crappy and thought maybe I missed something somewhere. I would like to ensure I have my info correct.

 

[wholeheartedly]

This is why a thorough validation is necessary.

The test was done at the state health dept, so I’m hoping they did a pretty thorough validation. Unfortunately I don’t have access to that data.

 

[Allegri]

Why is the person convinced the positive is a FP? Much more likely that some of the negatives in the group tested are FNs.

Really? I would think the opposite. Again, the only number that matters in this situation is the PPV. If we assume there is a prevalence of about 30M cases in the US (very generous?), and a test with 80% sensitivity and 95% specificity: PPV = 68%, NPV = 98%. True negatives are thus much more likely than true positives.
If you assume 10M cases in the US, PPV drops to 35%.

But it all depends on prevalence in the population you're testing.

 

[Med Director New England]

Really? I would think the opposite. Again, the only number that matters in this situation is the PPV. If we assume there is a prevalence of about 30M cases in the US (very generous?), and a test with 80% sensitivity and 95% specificity: PPV = 68%, NPV = 98%. True negatives are thus much more likely than true positives.
If you assume 10M cases in the US, PPV drops to 35%.

But it all depends on prevalence in the population you're testing.

The population living in nursing homes, assisted living, prisons are a high risk group (i.e. a higher prevalence of disease compared to general population). Have you not seen some of the results of general screenings for Covid in these settings?

Also how do you come up with 95% specificity for pcr based Covid testing? The analytic specificity for the cdc assay and the various high thru-put LDTs based on the cdc method is much higher, > 99%. The majority of labs had no FPs when they brought these tests on line using the FDA EUA validation guidelines.

A FN is much more likely than a FP in this setting.

 

[Allegri]

The population living in nursing homes, assisted living, prisons are a high risk group (i.e. a higher prevalence of disease compared to general population). Have you not seen some of the results of general screenings for Covid in these settings?

Also how do you come up with 95% specificity for pcr based Covid testing? The analytic specificity for the cdc assay and the various high thru-put LDTs based on the cdc method is much higher, > 99%. The majority of labs had no FPs when they brought these tests on line using the FDA EUA validation guidelines.

A FN is much more likely than a FP in this setting.

These are good points. You may be right. But consider: the relevant situation is prevalence in nursing homes in which a case has not previously been identified. Most of the data you refer to about high prevalence in these settings is gathered by testing everybody once a positive case has already been identified, and the virus has had a chance to spread. I don't see as much data on the prevalence when such facilities are randomly tested (please post if someone knows some of those data). I have seen 4% in the homeless shelter study in Boston when tested prior to known positives being identified.

If we use 4% prevalence, 99.9% specificity, 80% sensitivity, we are still looking at NPV of 99% and PPV of 97% (FP more likely than FN). But really, it's not FP vs FN that matters here; rather it's FP vs TP. And clearly TP wins.

 

[Med Director New England]

These are good points. You may be right. But consider: the relevant situation is prevalence in nursing homes in which a case has not previously been identified. Most of the data you refer to about high prevalence in these settings is gathered by testing everybody once a positive case has already been identified, and the virus has had a chance to spread. I don't see as much data on the prevalence when such facilities are randomly tested (please post if someone knows some of those data). I have seen 4% in the homeless shelter study in Boston when tested prior to known positives being identified.

If we use 4% prevalence, 99.9% specificity, 80% sensitivity, we are still looking at NPV of 99% and PPV of 97% (FP more likely than FN). But really, it's not FP vs FN that matters here; rather it's FP vs TP. And clearly TP wins.

Take a closer look at your 2x2

A condition wirh4% prevalence with a test with 80% sensitivity and 99.9% specificity will have 8 x more FN than FP. I am not saying the Spec is that high (we don’t know for sure what it is) but just sticking with your example

 

[bauber]

The FDA published a calculator to help determine many of these parameters. Some of you may find it helpful.

https://www.fda.gov/media/137612/download

 

[Allegri]

Take a closer look at your 2x2

A condition wirh4% prevalence with a test with 80% sensitivity and 99.9% specificity will have 8 x more FN than FP. I am not saying the Spec is that high (we don’t know for sure what it is) but just sticking with your example

You are correct. But I think we are now talking about two different probability scenarios. Yours is #1, mine is #2.

Scenario 1: I am going to test a population. What is the probability that any given test result will be FN, or FP, or TP, or TN? As you say, that probability given the above test performance is TN>>TP>FN>FP.

Scenario 2: I have in front of me a positive result, and some negative results. What is the probability that positive result is a FP? That is 1-PPV. And what is the probability that a given negative result is a FN? That is 1-NPV.

 

[LADoc00]

See False positives in reverse transcription PCR testing for SARS-CoV-2
False positives can occur especially when prevalence is low

False positives arent CAUSED by low prevalence, they are what they are but in a low prevalence location you are more likely to encounter a FP than a TP, this is especially true for serology testing with a 5% FP rate and a community prevalence of 0.1%

But anyone else here have graduate education in molecular genetics?? How is the specificity of PCR only 95 or 99%? If you properly designed your primers for conserved regions absolutely unique to SARS-CoV-2 then the amplification should be 100% specific.....no way it is anything close to 95%

EDIT: just read the article. The FP rate is NOT an indictment of PCR as a technology but the failure of labs to clean instrumentation between runs. This is a common mode of attack of defense attorneys in drug testing and I have testified in such cases with regularity but with modern lab equipment this has never actually been well proven to occur. Is it possible? Of course. Is your test likely a FP? No.

 

[gbwillner]

....

But anyone else here have graduate education in molecular genetics?? How is the specificity of PCR only 95 or 99%? If you properly designed your primers for conserved regions absolutely unique to SARS-CoV-2 then the amplification should be 100% specific.....no way it is anything close to 95%

....

You rang????

You are correct that in theory PCR experiments are very, very specific. But the process of deploying a test in the real world can result in FPs from multiple sources:
1. pseudogenes or high homology between targets can result in FPs
2. improper or lax annealing temperatures/conditions
3. a host of pre-analytic/ analytic issues that result in contamination or other errors:
not sterile technique/re-using pipette tips
improperly laid out workflow/ not separating pre/post PCR areas
sample swapping


While most can and will be avoided, this is a service performed by humans, and like anything else, they have error. No test is perfect, this is just part of the reason why.

I do agree in general it is more likely in this scenario to have false negative tests than FPs, but only as stated before, this is not a low-prevalence contagion- it is a Pandemic, and I assume a good proportion of us have been exposed and don't even know it.

 

[wholeheartedly]

well there are definitely differences between analytic sensitivity and specificity vs clinical sensitivity and specificity.

i am in a very very low prevalence rural type area, so that is a factor.

but i’ve poured over the lit and the interwebs and everything so far is coming up 99-100%

I have no idea where this dude thinks he saw 70-80% specificity for PCR. I’m not finding it.


we tried to get information on the Ct values but the lab tech at state got defensive when our lab mgr. said there was concern about a false positive. said if it was we contaminated it at collection and no Ct values given.

 

[Allegri]

Analytic specificity for a PCR assay would be expected to be well north of 99%.
Clinical specificity brings in more potential issues as described by gbw above but should still be north of 95%.
If we are talking about the likelihood this positive result is a true positive (PPV), that could be as low as 80% if you assume:
- test with 80% sensitivity and 99% clinical specificity
- prevalence/pretest probability <5%

Unfortunately a lot of assumptions are being made without a lot of data.

More importantly, the risk of discounting this case as a false positive has enormous potential cost to the care center! If I'm in that situation, a positive should be assumed true until a repeat test disagrees.

 

[wholeheartedly]

Analytic specificity for a PCR assay would be expected to be well north of 99%.
Clinical specificity brings in more potential issues as described by gbw above but should still be north of 95%.
If we are talking about the likelihood this positive result is a true positive (PPV), that could be as low as 80% if you assume:
- test with 80% sensitivity and 99% clinical specificity
- prevalence/pretest probability <5%

Unfortunately a lot of assumptions are being made without a lot of data.

More importantly, the risk of discounting this case as a false positive has enormous potential cost to the care center! If I'm in that situation, a positive should be assumed true until a repeat test disagrees.

Agree!

 

[Vicinal]

well there are definitely differences between analytic sensitivity and specificity vs clinical sensitivity and specificity.

i am in a very very low prevalence rural type area, so that is a factor.

but i’ve poured over the lit and the interwebs and everything so far is coming up 99-100%

I have no idea where this dude thinks he saw 70-80% specificity for PCR. I’m not finding it.


we tried to get information on the Ct values but the lab tech at state got defensive when our lab mgr. said there was concern about a false positive. said if it was we contaminated it at collection and no Ct values given.

Never understood how they set the Ct value In this type of non-quantitative test. Obviously you can’t run endless cycles. Having a low amount of target can confound any manufacturer prescribed Ct relative to provided controls ( likely 19-25 cycles) this does not mean target is not there. It is just the limit of this test based on whatever control is used and provided. It’s an RNA prep right? I’m more concerned about false negatives..Small amount of target that would be amplified outside of the “recommended“ Ct. This may be clinically irrelevant but have not seen any data on this to know definitively. How much virus do you need to be infectious? If there is target detectably amplified after 40 cycles does that matter?

 

[wholeheartedly]

Never understood how they set the Ct value In this type of non-quantitative test. Obviously you can’t run endless cycles. Having a low amount of target can confound any manufacturer prescribed Ct relative to provided controls ( likely 19-25 cycles) this does not mean target is not there. It is just the limit of this test based on whatever control is used and provided. It’s an RNA prep right? I’m more concerned about false negatives..Small amount of target that would be amplified outside of the “recommended“ Ct. This may be clinically irrelevant but have not seen any data on this to know definitively. How much virus do you need to be infectious? If there is target detectably amplified after 40 cycles does that matter?

I’m not sure. But if the Ct value is high the thought is it’s more likely to be contamination being amplified. A Clin micro expert at my old shop recommended if we were concerned about a false pos to get the Ct value and consider retesting the same sample as a starting point.

I have no idea how the threshold level is set, but it’s a very interesting question.