I have analyzed the RISE data that is put out every year. Some data has shown a correlation between a score above 500 and passing the boards, but this is weak, IMHO. If you look at their data, there is no significant difference in RISE scores between 1st, 2nd, 3rd, or 4th year residents. I think that tells you all you need to know about this test.
I think it would be an understatement to say that there is no significant difference in RISE scores between 1st, 2nd, 3rd, or 4th year residents. If you look at the overall score and each section scores, you would see that there's a 30-40 point increase as you go up the year level. The largest increases tend to be in subjects that require rotation exposure/experience in order to master, e.g. Hemepath, cytology, and surgical pathology. Conversely, in areas where the subject is not specifically instructed during residency (like lab managment), one sees only a small bump in score as you go up the PGY level. A 30-40 point increase each year may not seem significant, but as the RISE scores are curved, a 30-40 point difference near the median score actually represents quite a few additional questions that one needs to answer correctly. By comparison, if you're at the upper or lower extreme of the curve, then one or two questions that you answered correctly/incorrectly may actually result in a much larger difference in the score.
And I've always heard about the magic score of 500 that has been used to predict board success/failure. I have tried to search online for any published sources to see if there has actually been a study, but could not find anything. According to actual data published by the ASCP, it is much more meaningful to look at the percentile rank as opposed to the absolute score. They correlated PGY4's performance on the 2008 and 2009 RISE administration with their subsequent board passing rate. For the 2008 RISE, the top quartile, 2nd quartile, 3rd quartile, and bottom quartile had a subsequent board pass rate of 97%, 92%, 86%, and 46% respectively. For the 2009 RISE, the numbers were 100%, 99%, 94%, and 66%, respectively. So I think the moral of the story is, if you're above the mean on your PGY4 RISE, then there's only a very small chance (~3%) of failing the board. They stated that for PGY4 in the bottom quartile, only 1/2 - 1/3 passed all of their certifying exams. In addition, if a resident is in the lowest quartile as a junior, they are likely (2/3) to be in the lowest quartile for their senior RISE score.
Source:
http://www.apcprods.org/PRODS/documents/H_Rinder_USCAP_2012.pdf (page 17 of the document)
Also, according to page 15 of the document, nobody who scores 570 or higher on surgical pathology OR cytology fails their AP boards. Similarly, nobody who scores 600 or higher on hematology OR chemistry fails their CP boards.
Page 26 also has some interesting statistics: on average, programs that do not proctor the RISE exam (15% of all programs in 2010) only score 3 points higher than those that do (488 vs 485). A surprisingly small difference. Residents that take the exam unproctored do significantly better in surgical pathology and molecular, but worse in microbiology (!?), than those who the exam in a proctored setting.
