Are there many psychiatrists involved in psychometric research?
psychometrics
- Thread starter tehdude
- Start date
- Joined
- Oct 16, 2001
- Messages
- 2,871
- Reaction score
- 1,009
Depends on the psychiatrist and the measure. Many are developed by psychologists to become an intellectual property/copyrighted measure. Some psychiatrists develop them for outcomes research. The YBOCS, for example, or the MoCA.
So the answer is yes psychiatrists are involved, but depends on your definition of "many." Certainly not a majority.
So the answer is yes psychiatrists are involved, but depends on your definition of "many." Certainly not a majority.
In terms of clinical expertise and everyday use of psychometrics, its neuropsychology. Some clinical psychologists spend much of their careers in psychometric research as well. Kevin Mcgrew (an educational psychologist, I think) has a really neat blog that you could check out - http://www.iqscorner.com/. Of course, running a search on pubmed and psychinfo would be your best bet to see who is doing what.
It also depends on what specific population you want to look at - as obviously the neurobiology of typically developing intelligence is much different than that of those with intellectual disability. Even within the ID spectrum, wilsons presents much differently than downs syndrome, PDD, etc in terms of cognitive strengths and weaknesses.
It also depends on what specific population you want to look at - as obviously the neurobiology of typically developing intelligence is much different than that of those with intellectual disability. Even within the ID spectrum, wilsons presents much differently than downs syndrome, PDD, etc in terms of cognitive strengths and weaknesses.
There are a few researchers out there who work on the neurobiology of giftedness (not sure how many psychiatrists exactly, but there are some out there). I'm not sure who did this study, but one group used fMRIs and PET scans to investigate differences between children profoundly gifted in mathematics and more moderately gifted children during problem solving exercises.
PhDs tend to delve into this field more readily than MDs, but, especially for clinical research, there is a place for MDs, as well. One place of interest for you might be CTY at Johns Hopkins. They have a program called SET (study of exceptional talent), which is comprised of students scoring 700 or higher on verbal or/and mathematics subsections on the SAT. Many studies have been conducted using this sample (albeit, mostly from the social sciences as opposed to neurobiology)...
PhDs tend to delve into this field more readily than MDs, but, especially for clinical research, there is a place for MDs, as well. One place of interest for you might be CTY at Johns Hopkins. They have a program called SET (study of exceptional talent), which is comprised of students scoring 700 or higher on verbal or/and mathematics subsections on the SAT. Many studies have been conducted using this sample (albeit, mostly from the social sciences as opposed to neurobiology)...
I'm looking at this now. Hmm.. their psychometrician is a lawyer with a library studies degree. I guess then anyone can do it.
There's definitely a lot of sketchy studies out there. Dr. Lubinski (PhD) has done some psychometrics and studies of the SET population. Dr. Silverman (Ed.) and Dr. Gross in Australia (PhD) have also studied the upper ends of intelligence (not SET, although I think some of their samples qualified for it). There is a definite need for MDs in this area to increase credibility of studies (especially the unqualified researchers) and the field in the medical community. You'd probably be able to carve out your own niche...
- Joined
- Feb 8, 2004
- Messages
- 8,026
- Reaction score
- 4,176
All things being equal you will likely get more structure and training in the field of psychometrics in a psychology curriculum. Most psychiatry residencies are more about treating patients than teaching academics and theory that are not used in every clinical practice. As my fellowship PD put it. "Oh there's the SCID, how nice that we have tools to make our field look legitimate but no one ever uses it in real practice." (Just in case you didn't get it, he was making a criticism of our lack of knowledge and use of psychometrics).
There are M.D.s working on psychometric scales. The reality, however, is that these people often went into this type of research above and beyond the norm of what is expected or taught in a typical residency program.
If you are looking to do work in this area and want to do it through psychiatry, you're going to have to look into programs that are going to allow you to explore these horizons. Many will not. In fact most programs will be littered with attendings who never once used a HAM-D or other scale other than perhaps an AIMS and when you ask them to teach you about it they'll be clueless, perhaps even a bit insulted because big attending is always supposed to feel he has the answers to everything. Otherwise you're going to do it on your own by reading up on it without having an attending to further guide you or wait until fellowship and get into one that will allow you to do these things.
Yeah, I know I'm sounding a bit cynical, but unless you've been extremely lucky you know what I'm talking about. As for you medstudents and residents, I've had this happen to me while I was in your position, and I'm seeing it from the opposite side. I'm an attending and I see several of my colleagues pulling this type of attitude with people they should be better teaching.
There are M.D.s working on psychometric scales. The reality, however, is that these people often went into this type of research above and beyond the norm of what is expected or taught in a typical residency program.
If you are looking to do work in this area and want to do it through psychiatry, you're going to have to look into programs that are going to allow you to explore these horizons. Many will not. In fact most programs will be littered with attendings who never once used a HAM-D or other scale other than perhaps an AIMS and when you ask them to teach you about it they'll be clueless, perhaps even a bit insulted because big attending is always supposed to feel he has the answers to everything. Otherwise you're going to do it on your own by reading up on it without having an attending to further guide you or wait until fellowship and get into one that will allow you to do these things.
Yeah, I know I'm sounding a bit cynical, but unless you've been extremely lucky you know what I'm talking about. As for you medstudents and residents, I've had this happen to me while I was in your position, and I'm seeing it from the opposite side. I'm an attending and I see several of my colleagues pulling this type of attitude with people they should be better teaching.
Here are a few research residencies in psychiatry (not sure about psychometrics, but focus on research):
Northwestern University Medical School - Physician-Scientist Training Program - Psychiatry University of California San Diego School of Medicine - Research Residency Training Track - Department of Psychiatry - Psychiatry University of Colorado at Denver - University of Colorado Psychiatry Research Track - Psychiatry - Aurora, CO Yale University School of Medicine - Albert J. Solnit Integrated Training Program - Department of Psychiatry - Psychiatry
From:http://www.physicianscientists.org/careers/training/residency
Northwestern University Medical School - Physician-Scientist Training Program - Psychiatry University of California San Diego School of Medicine - Research Residency Training Track - Department of Psychiatry - Psychiatry University of Colorado at Denver - University of Colorado Psychiatry Research Track - Psychiatry - Aurora, CO Yale University School of Medicine - Albert J. Solnit Integrated Training Program - Department of Psychiatry - Psychiatry
From:http://www.physicianscientists.org/careers/training/residency
I'm looking at this now. Hmm.. their psychometrician is a lawyer with a library studies degree. I guess then anyone can do it.
Anyone can be trained to give a neurological exam...but the best person to interpret the neurology exam is a neurologist. The same is true for neuropsych tests, you can train anyone to give them, but the interpretation requires a clinical license. I was a psychometrician before I went on to my doctoral program, which means that I was supervised by a neuropsychologist who selected and interpreted the tests I administered based on patient characteristics, floor and ceiling effects, sensitivity to a diagnosis, and usefulness for making a differential diagnosis.
Last edited:
Can you be more specific? Whats your definition of a sketchy study...I'm just curous about what you're getting at here.
I don't remember specifics, but some studies are more qualitative (not using psychometrics or any testing at all or ignoring what's there) than quantitative or are performed by someone unqualified to interpret the data (that lawyer if he/she is doing the interpretation as well, nonstatisticians doing the statistics...). It's just something to watch out for if you're not familiar with a specific psychology journal. But that's just a statistician's take on it...
I don't think that is a fair comparison. A neurologist has to complete 4 years of medical school and 4 years of internship/residency. In contrast, working as a psychometrician apparently doesn't require any license or training. Even graduate quantitative psychology degrees that are geared specifically for such jobs have few prerequisites, and sometimes the coursework can be completed entirely online in a relatively short period of time. There aren't many things that you can't learn on your own, and I don't think interpreting SAT scores is an exception.
Lil Mick,
The lawyer at the place you referenced is not doing the interpretation because it would be, well, against the law. Its probably a second career thing or something for that individual and theres nothing wrong with that.
I was hoping for something more specific, like something you might find in the methods section of the studies you referred to...was the study overpowered with a huge sample size, or too small of an N with over-generalization of findings, inappropriate use of a particular measure, etc??
Correct me if I'm wrong, but I got the sense that you were implying that if a study doesnt have neuroimaging, its in the realm of social science and therefore not valid. Sally Shaywitz (a neurologist i think, definitely a physician of some kind) has done great imaging work in the field of dyslexia, but she would tell you herself that everyday clinical use of neuroimaging to diagnose a learning disorder would be absurd. Why spend thousands of our patients $$ when we have at least 2 tests i know of that, in 30 minutes or less, can diagnose a reading disorder with > 90% sensitivity and specificity regardless of the kids general intellectual ability.
In terms of superior intelligence, for the OP, let me recommend this book:Misdiagnosis And Dual Diagnoses Of Gifted Children And Adults: ADHD, Bipolar, OCD, Asperger's, Depression, And Other Disorders. I know one of the authors personally, and it would be a good start to understanding this population, psychometrics and all.
The lawyer at the place you referenced is not doing the interpretation because it would be, well, against the law. Its probably a second career thing or something for that individual and theres nothing wrong with that.
I was hoping for something more specific, like something you might find in the methods section of the studies you referred to...was the study overpowered with a huge sample size, or too small of an N with over-generalization of findings, inappropriate use of a particular measure, etc??
Correct me if I'm wrong, but I got the sense that you were implying that if a study doesnt have neuroimaging, its in the realm of social science and therefore not valid. Sally Shaywitz (a neurologist i think, definitely a physician of some kind) has done great imaging work in the field of dyslexia, but she would tell you herself that everyday clinical use of neuroimaging to diagnose a learning disorder would be absurd. Why spend thousands of our patients $$ when we have at least 2 tests i know of that, in 30 minutes or less, can diagnose a reading disorder with > 90% sensitivity and specificity regardless of the kids general intellectual ability.
In terms of superior intelligence, for the OP, let me recommend this book:Misdiagnosis And Dual Diagnoses Of Gifted Children And Adults: ADHD, Bipolar, OCD, Asperger's, Depression, And Other Disorders. I know one of the authors personally, and it would be a good start to understanding this population, psychometrics and all.
Last edited:
You are completely misunderstanding my point. I've worked with neurologists clinically for 3 years and am doing research with one right now...so I know what their training entails. All a psychometrician does is administer the darn tests. Thats it. There are no graduate degrees geared to prepare one to be a psychometrician.
Also...SAT testing? Where did you get that...there is no interpretation for that and is not something a clinical psychologist would give. The only thing I've ever had to do with standardized testing of that sort is make schools give accomodations to students with disabilities based on the results of our testing...which is a completely different animal and IS NOT something you can "train yourself" to do.
Last edited:
- Joined
- Oct 16, 2001
- Messages
- 2,871
- Reaction score
- 1,009
Physicians have clinical licenses, and can easily (if they choose) get training in administering and interpreting any psychometrics. Most don't choose to pursue that training, though.
So you think state governments prohibit all testing and interpretation by anyone who is not a licensed psychologist?
Merriam-Webster medical dictionary defines psychometrician as "a person (as a clinical psychologist) who is skilled in the administration and interpretation of objective psychological tests."
You can go to any job site and search for psychometrician. Most jobs require at least a masters in psychological or educational measurements, quantitative psychology, or statistics.
You can also look at universities that offers psychometric and quantitative psychology tracks.
The SAT is what the Center for Talented Youth uses as a screening exam.
The verbal section of the SAT is as highly g-loaded as anything else. Six of one, half a dozen of another.
Right, my point was that psychometricians do not have that license. Thanks for clarifying. I brought it up only because the OP seemed to be confusing the ability to give the test with the ability to asses, which are two different things especially when it comes to cognitive assessment.
tehdude,
In regard to the ability to interpret a test for the purpose of making a diagnosis, merriam webster is flat out wrong. And yes, its also under your purview as a psychiatrist...I never said it was not. Whopper's comment, however, is very worth noting.
As for the SAT, lots of places like that (CTY at hopkins) use different screening tools as broad cut-offs becuase they have to, but it doesnt mean the tests offer anything of clinical value. g itself accounts for only about 40% of achievement in academia. Clinically, the concept of g is not that useful. At the neurology clinic, for example, I'm much more concerned about assessing things like working memory, processing speed, and executive function than their scores on any achievement test. This is with kids with epilepsy, brain tumors, brain injuries, ALL, etc, but gifted kids are not immune to these problems. Achievement tests like the Woodcock-Johnson are more of what you would see in everyday clinical practice, because they parse out relevant cognitive variables (i.e. working memory ability is controlled for on reading comprehension tests), and its easier to find a neurologically impaired kids true level of achievement...gifted or not.
In regard to the ability to interpret a test for the purpose of making a diagnosis, merriam webster is flat out wrong. And yes, its also under your purview as a psychiatrist...I never said it was not. Whopper's comment, however, is very worth noting.
As for the SAT, lots of places like that (CTY at hopkins) use different screening tools as broad cut-offs becuase they have to, but it doesnt mean the tests offer anything of clinical value. g itself accounts for only about 40% of achievement in academia. Clinically, the concept of g is not that useful. At the neurology clinic, for example, I'm much more concerned about assessing things like working memory, processing speed, and executive function than their scores on any achievement test. This is with kids with epilepsy, brain tumors, brain injuries, ALL, etc, but gifted kids are not immune to these problems. Achievement tests like the Woodcock-Johnson are more of what you would see in everyday clinical practice, because they parse out relevant cognitive variables (i.e. working memory ability is controlled for on reading comprehension tests), and its easier to find a neurologically impaired kids true level of achievement...gifted or not.
Last edited:
I think this disagreement may stem from the ambiguity of lacking a strict definition of the word.
Even if only 40%, I would wager there is no other single discriminator that approaches that value.
It may not be that clinically useful at the neurology clinic, but I think identifying children with special needs, including gifted learners, is very clinically relevant.
I've heard it both ways, and honestly I'm still developing my own opinion on g. The trouble I find is that when kids are given a diagnosis of borderline to mild intellectual disability and are in that 65-75 IQ range, they are often written off by schools as not able to learn and thus they recieve less in terms of services. The goal for me, then, is to not pay as much attention to overall intellect and find pockets of strength and weakness with other assessment tools to funnel their educational therapies through. It can be tough, but must be done.
Assessing overall intelligence is important, for sure. However, there are so many cogitive abilities washed together in that type of assessment and thus too many anatomical areas that you dont get information on (i.e. the planum temporal for dyslexia). I find it similar to saying someones body temp is normal when their hair is on fire but their feet are in a block of ice.
You also might be interested in some of the psychometric issues that go along with the most widely used cognitive assessment tool, the WISC-IV. While its probably the most researched instrument on a neurological population (and thus why I use it more often than not), the nature of its scaled score to standard score conversion makes it less accurate in differentiating the borderline and mild MR/ID range, which is problematic for many reasons, one of which I mentioned above. Its also not good for assessing a gifted population due to ceiling effects. These issues are addressed in great detail in the McGrew blog i referenced in an earlier post, if your curious.
Last edited:
There are some very small sample sizes (around N=10 or less for the highest levels of intelligence) for some the studies, as well as overgeneralizations and recruiting problems (especially as some of the highest levels of intelligence have very few people and can be hard to identify with current methods). Additionally, many of the studies of the highest levels of intelligence are qualitative in nature, which is a good start to understanding the population, but, with small samples, not much else can be tested. In addition, some of the newer tests don't distinguish among the highest levels of intelligence (usually need out-of-level exams or using the old SM-LM). Also, since there are ceiling effects on some of the newer exams, one needs to be sure that the person administering the exam is aware of this and that the score obtained may be quite a bit lower than the true score because a person ran out of test before reaching his or her maximum ability. This is a concern if someone conducting a study has not had experience working with the upper levels of human intelligence.
I'm not implying that you need neuroimaging. I simply wanted to point out some of its uses with this population, as the OP is a medical student intestered in neurobiology who may be interested in that route, which is a newer development in the field and certainly a good fit with neuroscience. Neuroimaging has many problems (poor resolution comes to mind), and isn't practical in most situations. However, with the population that he would like to study ("severely gifted"), it may be a useful tool to add another piece of the puzzle of how these minds work.
I am not trying to dismiss social sciences. I spent several years in the field (as well as in education) and think that it has quite a bit to offer. It is amazing that SAT scores of children have such a high correlation with future careers/earnings/honors decades in the future (coming from studies of the SET population). However, some children get bored taking the exams and decide to, say, map out constellations on their bubble sheet during standardized tests or make patterns with IQ test questions (creative answering or running totals in math...) that suggests that many methods may need to be used to identify this population.
Misdiagnoses is a good book on the topic. Other books that OP might find interesting (case studies) are Exceptionally Gifted Children by Miraca Gross and Five Levels of Gifted by Deborah Ruf... Both deal with some of the problems of identification (current tests, creative answering during tests...) and qualitative differences at the higher ends of intelligence. The published studies on the SET population are good quantitative research, as well.
This is a good point. An organization I am consulting with right now does a lot of gifted testing for more than one school, and has used a test that is in no way appropriate for that population due to ceiling effects, among other things. Drives me nuts.
From where and when could they get this training?
- Joined
- Feb 8, 2011
- Messages
- 3
- Reaction score
- 0
IT514
I would like to say i admire your intentions.
it is a cruel society that discards the broken cogs; as you say 'by limiting services to childeren deemed unable to learn'
through broad intelligence tests.
I would like to say i admire your intentions.
it is a cruel society that discards the broken cogs; as you say 'by limiting services to childeren deemed unable to learn'
through broad intelligence tests.
- Joined
- Oct 16, 2001
- Messages
- 2,871
- Reaction score
- 1,009
http://www.pearsonassessments.com/pai/ca/training/workshops/MMPI.htm
http://www.upress.umn.edu/tests/MN11sessions.html
Depends on what you want training in.
- Joined
- Oct 7, 2006
- Messages
- 22,674
- Reaction score
- 5,022
A good start, but far from sufficient to administer/interpret an MMPI or related assessment.
- Joined
- Oct 16, 2001
- Messages
- 2,871
- Reaction score
- 1,009
I don't dispute that.
- Joined
- Feb 8, 2004
- Messages
- 8,026
- Reaction score
- 4,176
A class on psychometrics is offered in pretty much any college with a psychology curriculum. It is sometimes (though as far as I know not always) taught in a psychiatric residency but the reality is that several residents ditch the lecture due to being postcall, are half-asleep due to being postcall, don't pay attention to it because it's not going to be on USMLE Step III or the psychiatry board exam.
Not knowing psychometrics has no backlash/punishment effect for most psychiatrists.
IMHO, the difficult schedule in residencies mixed with attendings evaluating residents heavily on their clinical performance only (most attendings will not give a damn about a resident not studying so long as that resident effectively churns out patients because now the attending has to do less work) leads to this type of culture in psychiatry.
This is a shame because tests like the MMPI are of great help when you actually have some idea of how it's supposed to work. The MMPI among other tests for most psychiatrist are on the order of DBT. Its something you know is supposed to be employed during a specific situation and it's supposed to help in a particular way but hardly any psychiatrist knows how to do it.
IMHO, to fix this problem, psychometrics should be on the board exam (yeah right, like that's going to happen. You think guys on the board who don't know psychometrics are going to go back to school for this?). Another thing is residencies should start making their residents do psychometric scales. At. U. of Cincinnati, for example, I've seen attendings make their residents do the major scales (e.g. HAM-D, YBOCS., YMRS) on patients from day one of admission to discharge so the residents could see on a point scale the level of change day-to-day on a patient. A psychiatry residency should also actively have on hand psychologists who could do the psychometrics while explaining how it works to the resident. That too is done at U. of Cincinnati. All the time, the inpatient unit have particular patients where they want an MMPI or an HCR-20 (rates the risk of future violence), among several other scales done quickly, efficiently and with the psychologist explaining very well how to interpret the results. The same psychologists also do tests of malingering upon the request of the psychiatrist because the city provides funds for hospitals to address the issue (not something seen in most hospitals).
Not knowing psychometrics has no backlash/punishment effect for most psychiatrists.
IMHO, the difficult schedule in residencies mixed with attendings evaluating residents heavily on their clinical performance only (most attendings will not give a damn about a resident not studying so long as that resident effectively churns out patients because now the attending has to do less work) leads to this type of culture in psychiatry.
This is a shame because tests like the MMPI are of great help when you actually have some idea of how it's supposed to work. The MMPI among other tests for most psychiatrist are on the order of DBT. Its something you know is supposed to be employed during a specific situation and it's supposed to help in a particular way but hardly any psychiatrist knows how to do it.
IMHO, to fix this problem, psychometrics should be on the board exam (yeah right, like that's going to happen. You think guys on the board who don't know psychometrics are going to go back to school for this?). Another thing is residencies should start making their residents do psychometric scales. At. U. of Cincinnati, for example, I've seen attendings make their residents do the major scales (e.g. HAM-D, YBOCS., YMRS) on patients from day one of admission to discharge so the residents could see on a point scale the level of change day-to-day on a patient. A psychiatry residency should also actively have on hand psychologists who could do the psychometrics while explaining how it works to the resident. That too is done at U. of Cincinnati. All the time, the inpatient unit have particular patients where they want an MMPI or an HCR-20 (rates the risk of future violence), among several other scales done quickly, efficiently and with the psychologist explaining very well how to interpret the results. The same psychologists also do tests of malingering upon the request of the psychiatrist because the city provides funds for hospitals to address the issue (not something seen in most hospitals).
Last edited:
- Joined
- Oct 16, 2001
- Messages
- 2,871
- Reaction score
- 1,009
Our clinical measures used on our inpt services include the SLUMS, PCL-civilian, YBOCS, AiMS, YBOCS, YMRS, HAM-D. I've used the QIDS, COWS, and MoCA often as well.
- Joined
- Feb 8, 2004
- Messages
- 8,026
- Reaction score
- 4,176
Perhaps we should make a list of programs that actively tell their residents to use scales? I can tell you where I did general residency, scales were not actively being used except for AIMS.
- Joined
- Oct 7, 2006
- Messages
- 22,674
- Reaction score
- 5,022
I'm glad that some psychiatrists are interested in psychometrics, as they can be a nice complement to a clinical interview, H&P, etc. It's worth noting that psychometric research, screeners, psychological assessments, neuro assessments, etc....are all quite different. Something like a MoCA or Cognistat are fine for very basic data to support a referall for more in-depth assessment, but even as screeners they leave a lot to be desired. I get somewhat concerned about the "interpretation" aspect of some of the measures, but used as intended they can be a useful addition to many cases.
- Joined
- Oct 16, 2001
- Messages
- 2,871
- Reaction score
- 1,009
T4C,
while I agree, I also think it's important to recognize how incomplete many commonly used screening instruments used in medical practice are. Most still use the MMSE, for example. So switching to the SLUMS or MoCA is a decent step up for bedside/office screening instruments.
The interpretation aspect is something that has enraged me on multiple occasions while working in the school system this year. I couldnt believe one report I saw from a school psychologist (masters level) who used the bender-gestalt and concluded that there were no cognitive deficits based on the results of that test, and thus no need for further testing apart from a WISC and WIAT. Unfreakin believeable. I saw the kid 3 years after that report and we diagnosed dyslexia, with some executive impairments stemming from a previous neurological injury (I'm trying not to be too specific here about the case) that was documented in plain view in his chart. We recommended a wilson program and designed some behavioral/adaptive compensatory techniques for his frontal deficits and bam hes doing much better...but 3 years late due to incompetent use of a crappy measure.
Last edited:
- Joined
- Oct 7, 2006
- Messages
- 22,674
- Reaction score
- 5,022
Poorly interpreted data can lead to scary Dx's/Recommendations, though I'm not totally against the implementation of basic screeners used to flag possible issues. I worked in a VA where every new patient who came through Primary Care was administered a PHQ-9, a 4-5Q screener for PTSD, and a substance abuse checklist by the nurse. The screeners/checklists were straight forward, and no intepretation was needed because there were simple cutoff scores. If someone got flagged via a cut-off score, the health psychologist would follow up during that visit and see if there was a need for additional services. If there was a need for a psych med consult, psychiatry would be consulted. It freed everyone up to do what they do best.
nitemagi, I completely agree with you that many of the common screeners used in medical practice are not very good, and that going to a SLUMS or MoCA is a relative improvement. My concern is how the data is used from the screener. I've listened to some ridiculous Dx's/Recommendations based off of screeners like the SLUMS, Cognistat, MoCA, etc. The GOAT (Galveston Orientation & Amnesia Test) is the current bain of my existance, as I've heard providers try and make a Dx based soley on data from it. It is a very limited measure, but since that is the one assessment someone learned, that is what they rely on.
Reading this, it seems several different roles are being confounded.
1. Test development
2. Ensuring the test has adequate psychometric properties
and
3. Proper administration and interpretation of said test
I know several psychiatrists who do #1, but rely on psychologists for #2. #2 requires a firm understanding of the different types of validity and reliability, advanced statistics, etc. I don't think things like item response theory methods, other methods of analysis of latent constructs, ROC analysis, etc. are routinely covered in psychiatry training.
Depending on the test, #3 could be covered by a psychologist, psychiatrist, or simply a psychometrician (who would administer but not interpret). For example, a psychiatrist could easily be trained to administer clinical tools like the SCID or HAM-D. But there's probably less room in psychiatry to be trained to administer certain cognitive tests and to delve into the nuances of interpreting their raw scores.
1. Test development
2. Ensuring the test has adequate psychometric properties
and
3. Proper administration and interpretation of said test
I know several psychiatrists who do #1, but rely on psychologists for #2. #2 requires a firm understanding of the different types of validity and reliability, advanced statistics, etc. I don't think things like item response theory methods, other methods of analysis of latent constructs, ROC analysis, etc. are routinely covered in psychiatry training.
Depending on the test, #3 could be covered by a psychologist, psychiatrist, or simply a psychometrician (who would administer but not interpret). For example, a psychiatrist could easily be trained to administer clinical tools like the SCID or HAM-D. But there's probably less room in psychiatry to be trained to administer certain cognitive tests and to delve into the nuances of interpreting their raw scores.
