A diagnostic test for schizophrenia?

[Oceanview]

---

Members do not see this ad.

What do folks think of the utility of MRI in confirming the diagnosis of schizophrenia? (see article below)

Granted it's not ready for clinical implementation, but with 81% accuracy already, are we not far off?

Is this a useful allocation of resources, esp considering we lack the ability to cure the disease and our best treatments are non-specific and frought with issues of non-compliance?


Nature 438, 407 (24 November 2005) | doi:10.1038/438407a

Software shakes up schizophrenia diagnosis

Jennifer Wild

Brain scan analysis could reveal disease before symptoms.

Computer analysis of brain images can diagnose schizophrenia in patients, possibly even before symptoms arise, say researchers at the University of Pennsylvania in Philadelphia.

The technique, which is based on the ability of computers to tease out subtle differences between brain images, has split neuroscientists, with some questioning the value of the information produced.

But Ruben Gur and his colleagues are convinced it works. Earlier this year, they claimed that they could use the technique to detect whether individuals are lying or telling the truth (see Nature 437, 457; 2005). Now they have turned their attention to mental disorders.

They used magnetic resonance imaging (MRI) to scan the brains of 69 schizophrenia patients and 79 healthy controls. The images were analysed by computer to produce an algorithm that could tell the two groups apart. Rather than focusing on specific areas of the brain thought to be affected by the disorder, as has been tried in the past, they looked for subtle changes across the whole brain.

This type of approach has proved successful before — but only for images used to derive the algorithm. As soon as fresh images were introduced, the success rate plummeted. But this time the researchers say that they have overcome this problem and that they were able to classify new individuals as schizophrenic or healthy with 81% accuracy (C. Davatzikos et al. Arch. Gen. Psychiatry 62, 1218–1227; 2005).

Gur makes bold claims for the technique, saying that it is ready to be used alongside clinical histories and interviews to help diagnose schizophrenia. And, because many of the people studied were 18 years old and in the early stages of the disease, he believes that it might be possible to use brain imaging to diagnose the disease before classic symptoms appear.

"Now we can give a computer a picture of a person's brain and ask whether or not this person has schizophrenia," says Gur. "This should do for schizophrenia what the echocardiogram did for heart disease."

Other experts are much more cautious. "The results are interesting and promising, but need replication," says Philip McGuire, an expert in brain scanning and schizophrenia at the Institute of Psychiatry in London.

"Finding differences compared with healthy people is the easy part," says Chris Frith, who specializes in schizophrenia and imaging at University College London. What will be useful, he says, is a way to distinguish people with schizophrenia from those suffering from related problems such as mania or severe depression.

Gur and his team now plan to compare schizophrenics with other mentally ill people. They also want to compare young sufferers with their family members

 

[50960]

81% percent accuracy is equal to a p>15 which is nowhere near anything remotely diagnostic; let alone this was one study with a small sample. The rest of the flaws should be obvious.

 

[i61164]

Where do you get p > 15?

Anyway, the idea seems pretty interesting. The more scans they get in their database, the more accurate it is likely to become.

 

[Oceanview]

I'm no statistician, but I believe you are misinterpreting the meaning of a p value. The p value is the probability of obtaining a result as extreme or more extreme than that observed if the null hypothesis (i.e. no difference between groups) is true. Now, I'm not sure I know what that means...

But I think the relevant statistic in determing the utility of a diagnostic test is it's sensitivity and specificity.

For instance, according to Wikipedia the sensitivity and specificity of cardiac stress tests are:

Treadmill test: sensitivity of 67%, specificity of 70%
Nuclear test: sensitivity of 81%, specificity of 99%

Of course we have a gold-standard in cardiac angiography (while some still argue the significance of sub-critical stenosis). The point is we're using diagnositic tests that are less than 100% sensitive and specific to make medical decisions. Clearly we have a ways to go, but first-breakers are likely to get a full organic work-up incudling MRI, aren't they? Wouldn't it be nice if that MRI told us they had a "functional" illness as well.

BTW, in this MRI study of schizophrenia they imaged 69 cases and 79 controls. Will look more closely at their findings in Arch Gen Psychiatry. 2005;62:1218-1227.

 

[Anasazi23]

But I think the relevant statistic in determing the utility of a diagnostic test is it's sensitivity and specificity.

For instance, according to Wikipedia the sensitivity and specificity of cardiac stress tests are:

Treadmill test: sensitivity of 67%, specificity of 70%
Nuclear test: sensitivity of 81%, specificity of 99%

You're right. In medicine, we're much more concerned with sensitivity vs. specificity, and use that as a basis for further clinical testing and investigation. 81% accuracy doesn't reflect which of the two we're talking about here, but it is an impressive result for an investigative test thus far. Hopefully it becomes even more promising.

Diagnostic testing in medicine is always evolving, and few tests reach very high sensitivity and specificity. Remember the VDRL, FTA, ect...always evolving for the better.

 

[50960]

No, the null hypothesis has to do with the expected results of your experiment as opposed to the assumption that there is no relationship between your variables. I am very well aware of what a P value is. You know Sazi is always telling me I can't really know psychopharm unless I have gone to med school, and I am starting to believe this in reverse in regards to psychiatry and research design and statistics......................😉

 

[Anasazi23]

No, the null hypothesis has to do with the expected results of your experiment as opposed to the assumption that there is no relationship between your variables. I am very well aware of what a P value is. You know Sazi is always telling me I can't really know psychopharm unless I have gone to med school, and I am starting to believe this in reverse in regards to psychiatry and research design and statistics......................😉

In medicine, we are interested in sensitivity and specificity for diagnostic purposes. More specifically, sensitivity, a good starting point, is routinely compared to a gold standard. In this case, there is none (unlike nuclear stress test or US for cholecystitis). Therefore, to say the test is crap at this juncture is premature.

Believe it or not, the USMLE makes sure that we know that the definition of sensitivity = number of true positives divided by the number of true positives + number of false negatives. This is separate still from positive predictive value. P values and null hypotheses are normally in the realm of experimental design, which includes an independent variable. Can one speak of p-values to any meaningful degree if you have no comparative group?

 

[50960]

There was a control group, and by speaking in p value language I was trying to simplify. I am not saying it is crap by any means, but this sorta reminds me of the drug company studies where they try to fudge the #'s to make clinical significance=statistical significance. I am sure this is at least in the realm of the future of diagnostic imaging in psych.

 

[worriedwell]

A p value is not relevant in this situation I don't think. Although I could be convinced otherwise.

It seems promising and simple enough for me. I haven't read the article but at face value, when somebody tells me that they can run a test and 80% of the time it correlates with truth (in this case, with the clinical picture which we must take as the "gold standard"), then I'm impressed that it may be a clinical tool in my arsenal but it would be preferable if there were more tools and tests. Consider all the random tests and risk factors for things like rule out MI or syncope and you'll realize that 80% is nothing to sneeze at. To determine sensitivity and specificity we'd simply have to know how many controls were actually labeled as schizophrenia brains and how many schizophrenia patients were labeled as normal brains.

The interesting question is when does imaging, or diagnostic tests in general, define the gold standard. In other words, cardiac cath is the gold standard because we trust our image to show stenosis of a vessel in the heart, we define coronary artery disease by the pathology of plaques causing stenosis, and then we use angiography as the test that represents "truth".

The work, also at Penn, by Dr. Hahn and colleagues regarding neuregulin also published in Nature I believe may begin to further shed light on the disease of schizophrenia from a pathophysiologic standpoint. At some critical point, our understanding of this undoubtedly organic brain disease will reach a level where we will decide that some test will be the "gold standard" of defining the disease. Or at the very least, in the near future it will remain a clinical diagnosis but with very powerful tests that provide strong correlative support.

This is important, because these schizoaffective disorders, psychotic depressions, bipolar disorders with psychosis etc which especially at first break are hard to differentiate between and from schizophrenia, will have objective data that helps us "define" diseases differently and then we can target treatment algorhythms at different disease states instead of just going after symptoms which is what we do now and which clearly has limitations. Also, the subtypes of schizophrenia may be able to be differentiated or confirmed on imaging. For example, it would be nice to know if your catatonic patient who is 18 years old is suffering from catatonic depression or schizophrenia, because (correct me if I'm wrong) but the one with the mood disorder stands a great chance at responding to ECT while the one with schizophrenia probably has a far less chance of receiving benefit.

Boy is this field exciting, and for those of us just getting into psychiatry, we are in for some truly awesome breakthroughs I believe.

Anybody agree or disagree?

 

[50960]

A p value is very commonly used with data like this, take a peek at your next CME presentation. Basically it means if you continued to sample the same poulation over and over again in a random way you would expect the correlation/relationship that you found with your data (here 80% success rate) to occur P=.05- 95% of the time, p=.01- 99% of the time etc... A p-value is actually a function of how rigid a range you want your results to be interpreted based up on design and sample size. With the data you presented you would have an R2 value of approximately .4 with a p<.05. So to simplify I made the appoint that if you assumed an 80% correlation with MRI and schizo based upon the data presented the P would have to have been set at about .20. The bare bones for statistical significance (rules out random error etc) is P=.05 and up. Many of the stats you see in drug company data boast P-values of P<.001 or higher, and they can do this because they have a large sample size.

 

[Anasazi23]

It's hard to believe that this discussion (and I'm contributing to it) has degraded into p value discussions. Anyhow, the concept of p value has its usefulness in disproving the null hypothesis: that your experimental results are not due to chance...at p<.05 specifically, that the results are less than 5% due to chance.

If we extrapolate this to medical diagnostic tests, we need to determine if a test is no good simply because it can 'only' detect schizophrenia, say, 90% of the time. For some medical diseases, the testing results are even much poorer than that. The same hold true for medications, after examining effect size, you don't need 95% of your population to respond to a med to get FDA approval. VNS proves this quite well. 😉

It's likely very premature for us to be putting so much stock in this one test, but I think that when it comes to a psychiatric diagnostic test, considering we have nothing (if you don't consider psychological testing), 80% is better than nothing.

 

[worriedwell]

thank you psici for the information, i understand you are trying to be helpful (although it comes across somewhat condescending). for your information we do learn the meaning of the p value in medical school, prior to medical school, and in residency when reviewing literature. i also understand that my interpretation may still require further rectifying. lets see if i can think through what you are implying...

the null hypothesis here is that the brains are not different in schizophrenia vs normal

a p value of .05 would indicate that 19/20 times, the result (which disproves the null hypothesis-- or says that there is a difference in brains) would not be due to random chance and that is arbitrarily defined to be "significant".

in our examply, you are saying that since only 80% of the time the test was "accurate", then this only implies a p value of .20 and thus fails to reach significance or fails to disprove the null hypothesis.

Here is some further info about the research..."The results of the study demonstrate that sophisticated computational analysis methods can find unique structural brain characteristics in schizophrenia patients, with a predictive accuracy of more than 83%. Recently, Davatzikos and his group announced that further analysis of this data with even more sophisticated classification methods achieved a 91% predictive accuracy for diagnosis of schizophrenia via MRI (MICCAI 2005 meeting, Palm Springs, CA)."

I pulled this from another article on statistics and pasted it below:

Clinical trials are the medical community's diagnostic tests. When a physician sends a patient for a conventional diagnostic test, he or she is asking whether a certain disease is likely to be present. Similarly, when the medical community designs and executes a clinical trial, it wants to know whether a certain treatment is likely to work. And just like a diagnostic test, a clinical trial has the potential for yielding an incorrect, erroneous result.

The traditional approach to reporting probability of error in clinical trials includes P values and statements of power. The P value is the probability that a positive trial result is just the result of chance, given an assumed truth of the null hypothesis, which is that the treatment does not work. Power refers to the ability of a trial to show that a treatment works, given that in truth it actually does. Although presenting P values and statements of power has become lore in reporting clinical trials, careful reflection shows that clinicians might not gain as much value from them as they might think.

When a clinician reads a diagnostic test report that is "positive," the next question is, "How likely is it that my patient has the disease?" That is, what is the likelihood that this positive report is in fact correct? As described by Bayesian theory, the likelihood of a correct result given a positive test result, or positive predictive value, is related not only to the sensitivity and specificity of the test . . .

It would go on to say it also requires the prevalence of the disease but that is not the point of my post.

I'm thinking of the MRI as a diagnostic test. As it becomes more improved (and it already sounds pretty good), there will be a value in the test that provides some sort of diagnostic predictive value which becomes clinically useful.

That is why, although I recognize that when one places a literal p value to the data, it may not "reach significance", but that does not simply discount this research. I note also that the editors of Nature (not by any means a crappy journal) seemed to agree that it was worthy of print.