I always thought that a CBT can only be compared with another CBT, and not traditional paper based testing. So if you took it before 98 or whenever they switch to CBT, it's different.
As far as CBT is concerned, yea it's content-based, meaning that a 215 this year, according to NBME is comparable to a 215 last year. That said, I think from 2001-2003 the average is all around 215, up or down a point. Only standard dev is getting wider. Used to be 15 points, now it's around 20 point or so. Used to be impossible to get over 260s, but 2 people from my class alone crashed 270 in 2002. I think it has a lot to do with increased number of formerly experimental questions (that are thrown out in previous administrations) that started counting for real scores in recent administrations, according to a faculty member who write pharmacology questions for NBME. This way, although the mean raw score is lower, it really makes those who know those esoteric questions stand out. Instead of punishing the new, lower raw score on average, the NBME appears to reward those with higher raw scores, thus you get a bigger spread.
