PhD/PsyD Impact of COVID-19 on Assessment and Future of the Field

[psychtnt]

Hello, I hope you are all safe and in good health. I am currently working as a research assistant for a company that creates/publishes psychological assessments. I have noticed that my company and others like it are producing more online assessment tools and have doubled down on these efforts in the wake of the Coronavirus pandemic. As an aspiring Clinical Psychologist, I am wondering what companies like mine can do to support the field in times like this. I would love to advocate for my profession from within my company. Though I work with many Clinical Psych PhD's, many of them no longer work in private practice or see patients so their experience of this crisis is limited to its impact on psychological research. I'm curious if there are any online continuing education materials or other tools that might be helpful at this time. As someone who plans to dedicate her life to psychology, I am also wondering how this might affect the field overall. I am sure there will be a higher demand for psychologists but might there be a surge in the use of teletherapy after this period? Thank you all for all that you are doing to address the vast mental health concerns that this crisis will undoubtedly bring up. I hope that you are all focusing on taking care of your own mental and physical health in addition to that of your patients.

---

Members don't see this ad.

 

[PsyDr]

Your company is going to increasingly make tests into “black boxes”; where professionals are prohibited from understanding how the output is created. Which they already tried to do in like 2013. And you can’t tell me they won’t try to again. When your company is successful at this, it will allow them to market IQ tests and the like to schools. Schools who allow computerized test administration to be proctored by teachers. That'll allow them to sell a lot more tests, and cut psychologists completely out of assessment. No one will say anything about the validity of Timmy cheating off of Jimmy's ipad. Sure, some parents will send their kids to a psychologist for a second opinion in a private practice because they don’t want a stupid kid. But why pay for that when the school gives it? That company is also one of the biggest purveyors of school books in the USA. It would be reasonable to assume they’re gonna try to integrate books sales with that stuff. And get the data to sent to centralized servers/ "the cloud" for data mining.

Legal doctrine will completely exclude such tests because “black box” methods don’t conform to the law; so psychologists’ expertise will continue to dwindle.

Or maybe a huge company isn’t going to try to maximize profits, and I’m just being a grump. What do I know?

 

[BuckeyeLove]

daubert hearings for yearsssssssss

 

[PsyDr]

daubert hearings for yearsssssssss

Daubert hearings that last about 5 minutes.


"Your honor, I have no idea how that score was created. I just punched the stuff into the ipad and it gave me a number and graph. I know you judges have a history of trying to independently score the Hare PCL, but please don't do that this time. And can we ignore Daubert criteria? Cause all of that would be super."

 

[psychtnt]

Your company is going to increasingly make tests into “black boxes”; where professionals are prohibited from understanding how the output is created. Which they already tried to do in like 2013. And you can’t tell me they won’t try to again. When your company is successful at this, it will allow them to market IQ tests and the like to schools. Schools who allow computerized test administration to be proctored by teachers. That'll allow them to sell a lot more tests, and cut psychologists completely out of assessment. No one will say anything about the validity of Timmy cheating off of Jimmy's ipad. Sure, some parents will send their kids to a psychologist for a second opinion in a private practice because they don’t want a stupid kid. But why pay for that when the school gives it? That company is also one of the biggest purveyors of school books in the USA. It would be reasonable to assume they’re gonna try to integrate books sales with that stuff. And get the data to sent to centralized servers/ "the cloud" for data mining.

Legal doctrine will completely exclude such tests because “black box” methods don’t conform to the law; so psychologists’ expertise will continue to dwindle.

Or maybe a huge company isn’t going to try to maximize profits, and I’m just being a grump. What do I know?

Wow, I was not expecting a response like this but I'm glad you brought this up. I don't work for a large company, I work for a family owned smaller company whose mission is to improve the assessment process for psychologists. While I'm sure there is an interest in IQ products, we are currently not investing in those (to the best of my knowledge) and are focusing on making psych assessment more efficient from the perspective of a school psych who needs to test many kids in a day. I asked this question because my company was started by psychologists and we hope to support psychologists in the field as best we can. I was just wondering if there is anything you would like to see from these companies. I am very sorry to hear that psychologist expertise is being undervalued! (I am currently saving for school so I want my expensive education to be respected!)

 

[PsyDr]

Wow, I was not expecting a response like this but I'm glad you brought this up. I don't work for a large company, I work for a family owned smaller company whose mission is to improve the assessment process for psychologists. While I'm sure there is an interest in IQ products, we are currently not investing in those (to the best of my knowledge) and are focusing on making psych assessment more efficient from the perspective of a school psych who needs to test many kids in a day. I asked this question because my company was started by psychologists and we hope to support psychologists in the field as best we can. I was just wondering if there is anything you would like to see from these companies. I am very sorry to hear that psychologist expertise is being undervalued! (I am currently saving for school so I want my expensive education to be respected!)

RC Cola isn't healthy for you, just because it isn't Coca Cola.

Think about this for a second. Psychologists who do assessments are paid by the hour, for face to face work. "Improving efficiency" by testing as many kids as possible in a day can only be accomplished by shortening tests, changing who can administer tests, or changing things to group administration. All of those are bad news. Short tests= lower psychometric properties. Teachers administering tests and group administration= psychologists out of a job, and BS psychometric properties. They've tried all of those 30-40 years ago. All unreliable. Since that time, the science has only created further support of the idea that such factors invalidate testing.

 

[psychtnt]

RC Cola isn't healthy for you, just because it isn't Coca Cola.

Think about this for a second. Psychologists who do assessments are paid by the hour, for face to face work. "Improving efficiency" by testing as many kids as possible in a day can only be accomplished by shortening tests, changing who can administer tests, or changing things to group administration. All of those are bad news. Short tests= lower psychometric properties. Teachers administering tests and group administration= psychologists out of a job, and BS psychometric properties. They've tried all of those 30-40 years ago. All unreliable. Since that time, the science has only created further support of the idea that such factors invalidate testing.

I hear what you're saying and you're totally right. By "efficient" I mean eliminating heavy easels and putting them online. And eliminating some of the paperwork by having online record forms. Having online materials cuts down environmental impact as well as shipping costs. Every test I have worked on still takes a substantial amount of time to administer and score.

 

[WisNeuro]

RC Cola isn't healthy for you, just because it isn't Coca Cola.

Think about this for a second. Psychologists who do assessments are paid by the hour, for face to face work. "Improving efficiency" by testing as many kids as possible in a day can only be accomplished by shortening tests, changing who can administer tests, or changing things to group administration. All of those are bad news. Short tests= lower psychometric properties. Teachers administering tests and group administration= psychologists out of a job, and BS psychometric properties. They've tried all of those 30-40 years ago. All unreliable. Since that time, the science has only created further support of the idea that such factors invalidate testing.

Devil's advocate, long testing isn't necessarily better. All depends on the situation. I still run across community "neuropsychologists" who are running 6 hour + batteries to do a basic dementia eval.

But in general, I agree. I have thus far, refused to buy or use any black boxed assessment instrument. I'd hope that we as a field could band together to put a stop to this, but I am not optimistic. Maybe I'm chicken littling, but I do not envy any of the people in grad school currently spending time in neuropsych studies.

 

[Justanothergrad]

Meehl seems relevant here

APA PsycNet

psycnet.apa.org

Who makes the cookbook matters.

 

[PsyDr]

I hear what you're saying and you're totally right. By "efficient" I mean eliminating heavy easels and putting them online. And eliminating some of the paperwork by having online record forms. Having online materials cuts down environmental impact as well as shipping costs. Every test I have worked on still takes a substantial amount of time to administer and score.

So you are advocating for administering tests online, not proctored by psychologist. Which is taking work away from us. Since that's not in the financial interest of psychologists, the motivation can only be in the financial interest of the publishers. Or is your company using a metric ton of money to go to CMS to change RVUs for testing to protect our income? Hint: they're not.

The "heavy" issue is basically nonsense. An enormous test manual still weighs less than an iPad.

The shipping cost justification is also pure nonsense. Ever see who pays for shipping? It's the psychologist. If I'm ordering $900 in tests, an extra $35 in shipping is a non issue. Pearson tried this justification when they tried for the black box. It was "too expensive" to ship the data. Which came on a DVD. Which costs about $0.70 to ship. Which psychologists pay for anyway.

The environmental impact is also nonsense. Go look it up. Even major publishing professional organizations no longer use the environmentally friendly argument when justifying reducing paper. Your company's "online record forms" require massive amounts of electricity. The computers stuff is all plastic which doesn't break down. And I'll bet they're data mining.

 

[psychtnt]

So you are advocating for administering tests online, not proctored by psychologist. Which is taking work away from us. Since that's not in the financial interest of psychologists, the motivation can only be in the financial interest of the publishers. Or is your company using a metric ton of money to go to CMS to change RVUs for testing to protect our income? Hint: they're not.

The "heavy" issue is basically nonsense. An enormous test manual still weighs less than an iPad.

The shipping cost justification is also pure nonsense. Ever see who pays for shipping? It's the psychologist. If I'm ordering $900 in tests, an extra $35 in shipping is a non issue. Pearson tried this justification when they tried for the black box. It was "too expensive" to ship the data. Which came on a DVD. Which costs about $0.70 to ship. Which psychologists pay for anyway.

The environmental impact is also nonsense. Go look it up. Even major publishing professional organizations no longer use the environmentally friendly argument when justifying reducing paper. Your company's "online record forms" require massive amounts of electricity. The computers stuff is all plastic which doesn't break down. And I'll bet they're data mining.

Whoa... I'm not advocating for anything. I was just genuinely trying to see how my company could help psychologist during these difficult times. When I said that the easels could be online that still means the psychologist is administering the test. Its just using an online tool that can be easier to use for some practitioners as an *option*. I thought it was nice to have a system online where a psychologist could access all of their manuals for ease of use but maybe not. This is all beside my original point and I'm truly not the person you should be arguing with. I understand your frustration, but I seriously just wanted to know what I could share with the people I work with that might support you all during the pandemic. I guess I won't get that here.

 

[Fan_of_Meehl]

Meehl seems relevant here

APA PsycNet

psycnet.apa.org

Who makes the cookbook matters.

I'm a big fan of that guy.

 

[AbnormalPsych]

There are some good points being made here. You should consider taking this thread's feedback to your company, I mean, if you are genuinely interested in supporting us.

 

[PsyDr]

Whoa... I'm not advocating for anything. I was just genuinely trying to see how my company could help psychologist during these difficult times. When I said that the easels could be online that still means the psychologist is administering the test. Its just using an online tool that can be easier to use for some practitioners as an *option*. I thought it was nice to have a system online where a psychologist could access all of their manuals for ease of use but maybe not. This is all beside my original point and I'm truly not the person you should be arguing with. I understand your frustration, but I seriously just wanted to know what I could share with the people I work with that might support you all during the pandemic. I guess I won't get that here.

Here's some things you could do. Seriously. If your bosses TRULY believe this is all okay, then these are non-issues. Go tell your bosses that they should:

1) Provide a BINDING contract between the publisher and the purchasing psychologist that ensures that psychologists' revenue, inflation adjusted, will remain the same for digital or standard administrations. Your publisher offers to pay for any difference in payments between digital and in person test reimbursement, ad infinitum.

2) Get them to publish tests with REAL test security for any digital administrations, as required by some state laws This includes active 360 degree webcams to avoid third party effects. And provide an annual independent audit of these practices.

3) Get them to independently fund research that demonstrates that digital administrations have the same psychometric properties as standard tests.

4) As part of the user agreements, GUARANTEE that there will be no data mining. They provide an annual independent audit of these practices. If they want to data mine, then the psychologist whose work is being used can opt in and be paid at his/her consultant rate.

5) As part of the user agreement, guarantee that the publisher accepts any and all liability for breaches of test security, breaches of confidentiality, etc. Indicate that any legal proceedings resulting from digital administration will be paid for by the test publisher.

6) Offer a reward of a 51% stake in the company for anyone that can prove to an independent panel that the publisher is marketing or selling tests in a manner that advocates: group administration, the interpretation of tests results by an individual that has not been seen by the psychologist, or attempts to make the test a "black box".

7) Provide an independent energy audit that compares the carbon footprint of digital administration to traditional methods.

I'm not frustrated at all. I'm just not going to help a company take my income away and increase my liability, so they can make more money.

 

[Fan_of_Meehl]

So you are advocating for administering tests online, not proctored by psychologist. Which is taking work away from us. Since that's not in the financial interest of psychologists, the motivation can only be in the financial interest of the publishers. Or is your company using a metric ton of money to go to CMS to change RVUs for testing to protect our income? Hint: they're not.

The "heavy" issue is basically nonsense. An enormous test manual still weighs less than an iPad.

The shipping cost justification is also pure nonsense. Ever see who pays for shipping? It's the psychologist. If I'm ordering $900 in tests, an extra $35 in shipping is a non issue. Pearson tried this justification when they tried for the black box. It was "too expensive" to ship the data. Which came on a DVD. Which costs about $0.70 to ship. Which psychologists pay for anyway.

The environmental impact is also nonsense. Go look it up. Even major publishing professional organizations no longer use the environmentally friendly argument when justifying reducing paper. Your company's "online record forms" require massive amounts of electricity. The computers stuff is all plastic which doesn't break down. And I'll bet they're data mining.

Not sure what it says about me as a person but--as a psychologist--I can safely say that I've never experienced a scintilla of angst regarding the 'environmental impact' of any psychological assessment activities I've ever done.

But I'm just not a very virtuous person and probably deserve to be bound, gagged, and tossed into an active volcano in the name of Gaia and baby seals.

Man...I'm getting old.

 

[AbnormalPsych]

#6

 

[Sanman]

Not sure what it says about me as a person but--as a psychologist--I can safely say that I've never experienced a scintilla of angst regarding the 'environmental impact' of any psychological assessment activities I've ever done.

But I'm just not a very virtuous person and probably deserve to be bound, gagged, and tossed into an active volcano in the name of Gaia and baby seals.

Man...I'm getting old.

Generally agreed, lol. Nor do I particularly care.

While I am okay with computerized assessments on a sealed computer in which a psychologist can access the data for the test and provide behavioral observations, I fail to see the benefit of online testing for professionals. In my experience with assessment, behavioral observations are as important as assessment scores and online administration reduces opportunity for 1:1 behavioral observation. That is to say nothing of the logistics of billing, etc. I am curious as to what the company is working on and what they claim as the benefit. I can't imagine software can be developed quickly enough to address this pandemic anyway. I also don't believe that this software will be free or low cost, so....what is the supposed benefit?

 

[Fan_of_Meehl]

Dehum

Generally agreed, lol. Nor do I particularly care.

While I am okay with computerized assessments on a sealed computer in which a psychologist can access the data for the test and provide behavioral observations, I fail to see the benefit of online testing for professionals. In my experience with assessment, behavioral observations are as important as assessment scores and online administration reduces opportunity for 1:1 behavioral observation. That is to say nothing of the logistics of billing, etc. I am curious as to what the company is working on and what they claim as the benefit. I can't imagine software can be developed quickly enough to address this pandemic anyway. I also don't believe that this software will be free or low cost, so....what is the supposed benefit?

Dehumanization. Profit. Control. Consolidation/ centralization of data. Panopticon. Technocratic/bureaucratic/administrative hostile takeover. Technologizing an activity for the sake of technologizing it while overhyping the presumed advantages and ignoring any disadvantages.

I'd do anything to go back to the paper charting system we used in grad school to replace the multiple software platforms we have to deal with today to document patient encounters.

Do people really think that these software 'tools' make them better therapists or their therapy more effective? I see the utility of some of the smartphine apps but, jeez, if an EMP hit our VA I'd probably be ecstatic.

 

[WisNeuro]

Do people really think that these software 'tools' make them better therapists or their therapy more effective? I see the utility of some of the smartphine apps but, jeez, if an EMP hit our VA I'd probably be ecstatic.

I'd disagree on chart review. When patients are terrible historians, either due to dementia, malingering, or other, it is sure helpful to be able to see a decade+ of records from encounters in mine and several other large systems in the state.

 

[Fan_of_Meehl]

I'd disagree on chart review. When patients are terrible historians, either due to dementia, malingering, or other, it is sure helpful to be able to see a decade+ of records from encounters in mine and several other large systems in the state.

A fair point. Personally, I'd sacrifice that convenience/advantage if I could do away with: (a) login to system (PIV card authentication has been fiddly as of late; (b) requirement of typing anywhere from 2 - 6 separate chart notes per encounter; (c) logging into and getting kicked out of VetLink (pt check in app); (d) logging into Mental Health Suite to write a separate 'treatment plan' requiring every element to be broken down into Problem, Goal, Objective, Intervention; and (e) logging into Behavioral Health Lab to enter questionnaire data. To say nothing of the 30 or more emails per day (mostly unnecessary) as distractions.

If the goal was to create an environment in which I could operate as an efficient and effective psychotherapist, I'd place money on a bet that the setup of a phone, pen/paper, a filing cabinet and well-stocked professional library would be far superior to the current setup.

Very little of the pointing, clicking, logging in/out, copy/pasting adds up to better performance as a therapist and most of it is just a drain on attentional resources and is very distracting. I mean, how did Beck, Barlow, et al. ever manage without Mental Health Suite or CPRS? They must have been abysmal practitioners.

 

[AcronymAllergy]

Dehumanization. Profit. Control. Consolidation/ centralization of data. Panopticon. Technocratic/bureaucratic/administrative hostile takeover. Technologizing an activity for the sake of technologizing it while overhyping the presumed advantages and ignoring any disadvantages.

I'd do anything to go back to the paper charting system we used in grad school to replace the multiple software platforms we have to deal with today to document patient encounters.

Do people really think that these software 'tools' make them better therapists or their therapy more effective? I see the utility of some of the smartphine apps but, jeez, if an EMP hit our VA I'd probably be ecstatic.

I 100% agree with this.

I also 100% believe that adding computerized aspects to testing has the potential to significantly benefit assessment. However, there's a difference between computer-assisted testing (e.g., the examiner hands the patient a tablet to complete a portion of a test, or records the patient's responses in a tablet that they're using) vs. entirely computer-administered testing. I would advocate for, and don't know that there's any way of avoiding, the former; I would protest against the latter.

The main issues are identified by PsyDr, including the whole "block box" problem, being forced to participate in data mining, and the inevitable push toward lower administration and interpretation qualifications because computers make things seem easy. A computer can certainly help with reducing scoring and calculation errors, improving accuracy and numbers/types of observations (e.g., response times), and even identifying patterns in responses and results. A computer should not be relied on to churn out interpretations, allow for large group administrations, or cut the actual psychologist (i.e., the expert) out of the picture.

Unfortunately, I worry that with the trend toward less and less emphasis being placed on assessment, future psychologists may actually be in favor of full-on computer-administered and interpreted testing due to not fully understanding the tests themselves.

 

[Fan_of_Meehl]

I 100% agree with this.

I also 100% believe that adding computerized aspects to testing has the potential to significantly benefit assessment. However, there's a difference between computer-assisted testing (e.g., the examiner hands the patient a tablet to complete a portion of a test, or records the patient's responses in a tablet that they're using) vs. entirely computer-administered testing. I would advocate for, and don't know that there's any way of avoiding, the former; I would protest against the latter.

The main issues are identified by PsyDr, including the whole "block box" problem, being forced to participate in data mining, and the inevitable push toward lower administration and interpretation qualifications because computers make things seem easy. A computer can certainly help with reducing scoring and calculation errors, improving accuracy and numbers/types of observations (e.g., response times), and even identifying patterns in responses and results. A computer should not be relied on to churn out interpretations, allow for large group administrations, or cut the actual psychologist (i.e., the expert) out of the picture.

Unfortunately, I worry that with the trend toward less and less emphasis being placed on assessment, future psychologists may actually be in favor of full-on computer-administered and interpreted testing due to not fully understanding the tests themselves.

"He/she/they who controls the algorithm...controls the world!"

 

[Therapist4Chnge]

Dehum
Dehumanization. Profit. Control. Consolidation/ centralization of data. Panopticon. Technocratic/bureaucratic/administrative hostile takeover. Technologizing an activity for the sake of technologizing it while overhyping the presumed advantages and ignoring any disadvantages.

I'd do anything to go back to the paper charting system we used in grad school to replace the multiple software platforms we have to deal with today to document patient encounters.

Do people really think that these software 'tools' make them better therapists or their therapy more effective? I see the utility of some of the smartphine apps but, jeez, if an EMP hit our VA I'd probably be ecstatic.

I'm *still* on a paper chart. We had a weird combo of chart + EMR. I still vastly prefer a chart bc EMRs just create so much useless information. Hey in-pt nursing note that is 2 lines and takes up 2 pages because of all of the useless copy over/pass through information.

 

[Fan_of_Meehl]

Right

I'm *still* on a paper chart. We had a weird combo of chart + EMR. I still vastly prefer a chart bc EMRs just create so much useless information. Hey in-pt nursing note that is 2 lines and takes up 2 pages because of all of the useless copy over/pass through information.

Right. A blank page forces you to organize your thoughts and strive to compose a coherent narrative, including relevant clinical hypotheses, data, observations and the interpretive logic upon which you conclusions are based. In reviewing electronic medical records, trying to isolate even a sentence or two of actual meaningful clinical observation or reasoning is like trying to find the proverbial needle in the haystack and is often frustrating and time consuming. If people were forced to generate content (and handwrite it out), they may be more motivated to think about what they want to write (prior to writing it) and strive to be concise, relevant, and incisive with their text.

 

[thebalmofhurtminds]

I would agree with this, if some peoples' handwriting wasn't abysmal.

 

[WisNeuro]

Way back when, I have been in places with handwritten notes, and also have had to review old scanned in handwritten notes. IMO, handwritten notes were just as bad as typed/dictated notes now. But, as balm has mentioned, at least I can read all of the poorly written notes now, instead of having to break out a fooking Rosetta Stone.

 

[Sanman]

Way back when, I have been in places with handwritten notes, and also have had to review old scanned in handwritten notes. IMO, handwritten notes were just as bad as typed/dictated notes now. But, as balm has mentioned, at least I can read all of the poorly written notes now, instead of having to break out a fooking Rosetta Stone.

True, but bad handwriting was built in malpractice protection. Can't sue me for what you can't read, lol!