- Joined
- Aug 20, 2010
- Messages
- 1,958
- Reaction score
- 2,076
I never said you couldn't, I said they would be less accurate.
The only people who truly know how it's done are those at the AAMC, so unless Jepstein works for them his argument is as valid as mine, and many others in here.
Just because you do a post test analysis does not automatically make it curved; it depends what type of analysis and what you do with it. You can still have a true scaled test with post-test analyses.
Here's an example. Using data from experimental questions a test is made along with initial conversion factors to create a scale which assumes reliability between it and previous tests. The test is administered and after using the initial conversion factors, which are based on experimental data, the data set of scaled tests scores is varied/skewed significantly from the data set of previously validated scaled scores to which it is being compared, bringing into question reliability given the comparable sample size. Thus the conversion factor would have to be reassessed so that reasonable reliability exists between tests. This would adjust for things such as context (likely a weaker effect in this case, but definitely present), test errors, etc. This isn't producing a curve, because it's not forcing the data to fit within a fixed distribution. Nor is it adjusting the scale, it's adjusting for reliability between tests. The only other option would be to say "hey, we messed-up so you'll have to re-take the MCAT so your score is reliable. But don't worry, we've worked the kinks out this time."
Last edited:
