EPPP Passing Rates and Adverse Impact

[rso19]

The actual passing rates for the EPPP are much lower than I initially thought they would be (a lot of outdated information from Google searches). Also wanted to start a discussion on the whether it is ethical for this exam to be in place as I was reading about a study "Are demographic Variables Associated with Performance on the Examination for Professional Practice in Psychology (EPPP)?"

The author, Brian Sharpless, PhD, is associate professor at the American School of Professional Psychology. To collect data, he used a Freedom of Information Act (FOIA) to request test results and demographics from the New York state board of psychology for its candidates.

Dr. Sharpless gathered data on 4892 applicants and first-time EPPP takers. He obtained “Records of all doctoral-level psychology licensure applicants from the previous 25 years with EPPP scores, gender, ethnicity, and degree type were requested.”

He found that Blacks had a failure rate of 38.50% and Hispanics had a failure rate of 35.60%. Whereas, Whites had a failure rate of 14.07% and Asians had a failure rate of 24%. (which demonstrates adverse impact for Blacks and Hispanics)

The differences in minority candidates’ selection rate violates what is known as the “four-fifths rule.” This means that the pass rate for minority groups fails to reach at least 80% of the pass rate for the majority group.

Typically, when a test has this impact, industrialorganizational psychologists exercise very careful methods to set cut scores, seek additional validity or research, and investigate possible replacements with less disparate impact. (ironically I learned about disparate impact and how it should be addressed while studying for the EPPP)

“… if the EPPP is found to lack acceptable validity evidence (or if a decision is made to not submit the measure to further empirical testing), then it will remain open to charges of being a potentially arbitrary barrier in an already protracted path to professional independence…”

This study was published on October 22, 2018. I have not heard any updates from the ASPPB on whether they have taken the appropriate steps to address this issue.

---

Members don't see this ad.

 

[WisNeuro]

I would like to see these results once program is controlled for. For example, do POCs from reputable programs differ from their peers at the same institutions. Otherwise, I would worry that this is a result that is almost wholly explained by the predatory nature of diploma mills. Which, would mean that there is still a racial disparity, but the problem may be in recruitment practices and poor training by diploma mills, rather than the EPPP.

 

[futureapppsy2]

Here is the study the OP mentions: Are demographic Variables Associated with Performance on the Examination for Professional Practice in Psychology (EPPP)? - PubMed

Interestingly, not a large difference in first-time pass rates between PhD and PsyD graduates (85% v. 80%).

 

[WisNeuro]

Do you by chance have the study, what is the sample drawn from?

 

[futureapppsy2]

Do you by chance have the study, what is the sample drawn from?

Linked the study above—looks like it’s drawn from all EPPP examines in one “populous state” (I’m guessing California). No apparent interactions between race/ethnicity and degree type.

 

[WisNeuro]

Linked the study above—looks like it’s drawn from all EPPP examines in one “populous state” (I’m guessing California). No apparent interactions between race/ethnicity and degree type.

That could be an issue. I think we'd need to delve into the data to see what the best solutions could be. For one thing, as I stated above, you'd need to control for program quality, as I imagine the mills/poor quality programs, heavily market to non-traditional students. In which case, the issue is not with the test, per se.

 

[ClinicalABA]

Linked the study above—looks like it’s drawn from all EPPP examines in one “populous state” (I’m guessing California). No apparent interactions between race/ethnicity and degree type.

It was New York. He replicated it in CT with similar results ( Sharpless, B. A. (2019, December 12). Pass Rates on the Examination for Professional Practice in Psychology (EPPP) According to Demographic Variables: A Partial Replication. Training and Education in Professional Psychology. Advance online publication. http://dx.doi.org/10.1037/tep0 ). I don't have access to the full paper from the OP, but can see this one. Sample is interesting- state provided data on ~4000 administrations, ultimate study sample was 642 unique first time test takers where gender and ethnicity was available. Of these, ~50% contained degree info (ph.d, psyd, or EDD). Sample sizes varied across ethnicity, with all data negatively skewed, thus he used non-parametrics (e.g. Chi-Squared analyses). He found that scores were significantly correlated with year of administration, with a negative slope in all ethnic groups since 2001 (first year available). I'll let you smart people interpret all that He does report that Psyds failed at a rate 3x that of Ph.D.s (not enough EdDs for analysis). Different than the NY study, Asians had a higher pass rate than Whites.

Here's that sample data from the 2019 study from CT that I cited above:

 

[futureapppsy2]

It was New York. He replicated it in CT with similar results ( Sharpless, B. A. (2019, December 12). Pass Rates on the Examination for Professional Practice in Psychology (EPPP) According to Demographic Variables: A Partial Replication. Training and Education in Professional Psychology. Advance online publication. http://dx.doi.org/10.1037/tep0 ). I don't have access to the full paper from the OP, but can see this one. Sample is interesting- state provided data on ~4000 administrations, ultimate study sample was 642 unique first time test takers where gender and ethnicity was available. Of these, ~50% contained degree info (ph.d, psyd, or EDD). Sample sizes varied across ethnicity, with all data negatively skewed, thus he used non-parametrics (e.g. Chi-Squared analyses). He found that scores were significantly correlated with year of administration, with a negative slope in all ethnic groups since 2001 (first year available). I'll let you smart people interpret all that He does report that Psyds failed at a rate 3x that of Ph.D.s (not enough EdDs for analysis). Different than the NY study, Asians had a higher pass rate than Whites.

Here's that sample data from the 2019 study from CT that I cited above:

View attachment 313288

Thanks. That table really highlights why this was perhaps not the best study to do with the CT data as the sample sizes for the non-White racial/ethnic groups are so small.

 

[psych.meout]

It was New York. He replicated it in CT with similar results ( Sharpless, B. A. (2019, December 12). Pass Rates on the Examination for Professional Practice in Psychology (EPPP) According to Demographic Variables: A Partial Replication. Training and Education in Professional Psychology. Advance online publication. http://dx.doi.org/10.1037/tep0 ). I don't have access to the full paper from the OP, but can see this one. Sample is interesting- state provided data on ~4000 administrations, ultimate study sample was 642 unique first time test takers where gender and ethnicity was available. Of these, ~50% contained degree info (ph.d, psyd, or EDD). Sample sizes varied across ethnicity, with all data negatively skewed, thus he used non-parametrics (e.g. Chi-Squared analyses). He found that scores were significantly correlated with year of administration, with a negative slope in all ethnic groups since 2001 (first year available). I'll let you smart people interpret all that He does report that Psyds failed at a rate 3x that of Ph.D.s (not enough EdDs for analysis). Different than the NY study, Asians had a higher pass rate than Whites.

Here's that sample data from the 2019 study from CT that I cited above:

View attachment 313288

In the original study, there were no significant differences in overall fail rates between PhD (~15%) vs PsyD (~20%) programs, but there were in the CT replication. This leads me to think that the real moderating factor is not degree type but funding and other practices associated with program quality (e.g., cohort size). Many NY PhD programs, even those at otherwise reputable institutions (e.g., NYU), tend to be unfunded or underfunded, which indicates less investment in the students and more stress to be training while accruing huge debt (e.g., >$40,000/year at NYU). Thus, I think there are fewer differences between NY PhD and PsyD programs, on average, than compared to other states, thereby obscuring differences between PhD vs PsyD and funded vs. unfunded/underfunded programs in general in the US. Compare this to CT, which has only 4 doctoral programs and only one of them is a PsyD and it's also the only one that isn't fully funded.

https://cdn.ymaws.com/www.asppb.net/resource/resmgr/EPPP_/2016_Scores_by_Doctoral_Prog.pdf

Two of the other 3 have 100% pass rates and the other had too few test takers to be registered. NY may be an outlier compared to most other states and probably not the best choice if you want a better picture of doctoral training and the EPPP overall.

 

[PsyDr]

eppp pass rate is also highly correlated with programs mean admission GRE scores and GPA scores. And Apa accreditation.

The pass rate for APA graduates is approximately 82%. The pass rate for unaccredited programs is approximately 55%.

Weird that this thread was posted from a new account...

 

[psych.meout]

eppp pass rate is also highly correlated with programs mean admission GRE scores and GPA scores. And Apa accreditation.

The pass rate for APA graduates is approximately 82%. The pass rate for unaccredited programs is approximately 55%.

Weird that this thread was posted from a new account...

And poorer quality programs and those that are unaccredited have lower standards for admission, leading to admission of applicants who aren't sufficiently prepared (and won't be prepared by their programs) or never will be (because they never should have been admitted to any doctoral program in the first place).

 

[borne_before]

And poorer quality programs and those that are unaccredited have lower standards for admission, leading to admission of applicants who aren't sufficiently prepared (and won't be prepared by their programs) or never will be (because they never should have been admitted to any doctoral program in the first place).

This could just be evidence of more systemic racism.

 

[WisNeuro]

I don't believe that anyone has claimed that there is no aspect of racism and/or racial disparity present here, just that the data does not include enough information as to where the problem lies. and subsequently where remediation efforts would be best utilized.

 

[JustNoticing]

This could just be evidence of more systemic racism.

Correct, it would be proof of systemic discrimination, which certainly exists, but wouldn't proof that the EPPP was the tool of discrimination. The question is where in the pipeline the discrimination is or is not occurring...

 

[foreverbull]

In addition to the above factors, I would also wonder about the intersectionality of SES and ethnicity here in terms of pass rates if they could collect data in terms of which class one was raised in. Just thinking about how those with more means live in more affluent school districts, could have had educational advantages throughout their earlier educational career (early interventions and/or tracking into gifted programs, more individualized attention, etc.), and including attending small private colleges in which there is more mentoring and guidance toward loftier career goals, and/or didn’t have to work part time jobs to get through school and/or could afford additional practice materials and/or tutoring, etc.

What we know from sociological data is that folks of color and particularly Black folks generally attend urban, underfunded schools that inadequately prepare them for college and beyond (same generally speaking for lower-SES folks too). I saw this firsthand teaching at a local CC and it was disheartening to see the effects of unequal public education based on districts/funding (and SES), which perpetuates systemic racism as a result.

 

[borne_before]

Correct, it would be proof of systemic discrimination, which certainly exists, but wouldn't proof that the EPPP was the tool of discrimination. The question is where in the pipeline the discrimination is or is not occurring...

Does anyone know if the professional schools admit more BIPOC (proportionally)?

 

[WisNeuro]

Does anyone know if the professional schools admit more BIPOC (proportionally)?

Significantly more so, particularly Black and Hispanic POC according to a 2017 report.

 

[Justanothergrad]

Agreed with others above. This study reminds me of what happens when great ideas don't get analyzed to the degree to which the data needs. This question of differential impact begs for a multiple moderation models and should likely be done utilizing national samples given the distribution of predatory programs (and that people leave and get licensed in states other than where they train). It would be unsurprising to find adverse impact in standardized testing outcomes consistent with what is observed, but the 'cause' is far from clear based on this data although I and many others may have suspicions about some of the cause.