I know Kaplan offers a free 3 hour Step 1 diagnostic test at their test centers to 'help you devise a study plan.' Seeing that I'm an only half way through my MS2 year (and have yet to cover many subjects tested on the boards), I can't see how this would be an accurate gauge of your baseline (since I can assume I'd have to guess on many, many answers). Is it supposed to just give you a sense of the questions, or actually test how much you remember? Has anyone taken this diagnostic test or have any input? Thank you!
Our school had some deal w/ Kaplan and administered us a Kaplan diagnostic. It's kind of interesting to see where you stand right now broken down subject by subject, and they give some very general prognostic advice - ie. >65% suggests high probability of passing, etc.
As you say, it's accuracy in predicting your weak areas is limited by the fact that you haven't covered all the subjects yet, but I do think it could be moderately helpful in 1)pointing out some basic holes you may not realize you have and 2) giving you some idea of how well you've retained material and thus how much work you have ahead of you.
If it's free and you want the practice, I'd do it, but I wouldn't get too hung up about it. IMO it's not going to show you anything about your weaknesses beyond what tracking your progress with a question bank will and if you really want to see where you're at in terms of numbers, the NBME practice tests seem like a much more peer-reviewed way to gauge yourself.