Step 1 questions that don't count

[pharaoh]

I have been unable to find out if step 1 trial questions are dispersed throughout the exam or if a whole 50 question block is composed of trial questions. Everybody I have talked to seems so sure about one or the other. Anybody know the truth? The reason I ask is after taking Step 1, it seemed that one block was so much more brutal than all the others to the extent that I almost didn't finish it.

---

Members don't see this ad.

 

[newbie5]

It is most like to be dispersed, otherwise ppl would just sleep through the block that wasn't going to count.

 

[Idiopathic]

Nobody knows...just like everything else about that godforsaken exam.

 

[chandler742]

I doubt that. First of all, no one has proof that there is experimental questions on Step 1 because no one from NBME has ever admitted this information.

Secondly, even if one block was experimental, how would you decide which one was experimental? Based on your premise, people would just sleep through that block. This is BS. Because there is no way for one to figure out which block is experimental. Do you really believe if your livelihood is on the line, you would try to figure out which block is experimental under the pressure of an examination. No body with a rational mind would attempt to do this unless they had some kind of tipoff. I believe the testmakers know this fact!!

Also, keep in mind that the GRE was computerized before the USMLE became computerized. The USMLE people used the computerized model of the GRE as proof that written exams can be replicated on a computerized form. In fact, USMLE are currently doing pilot projects for an adaptive USMLE. This will be inplemented within the next 5 years.

Lastly, I argue since the USMLE model the GRE in terms of computerized testing, it is possible that like the GRE, one section is experimental!! First, NBME followed GRE's lead to computerized testing. Secondly, it is following the adaptive test format. Therefore, I think there is real good chance that one of the blocks is an experimental block. The GRE has one experimental block. This is a proven fact.

Why would the USMLE try to reinvent the wheel and not follow the lead in this respect? If testmakers wanted to make a computerized exam, wouldn't they study the only computerized exam that is currently available( i.e. the GRE computerized exam) and copy the format. Of course they would. Think Bill Gates and Apple. In my mind, it doesn't make logical sense NOT to copy the format. I believe that NBME studied the GRE test, and made it similar. It takes too much effort to start from ground zero, when a proven computerized testing format already exists. It is more convenient for Step 1 to have one experimental block just like GRE. So I believe this is how it is.

As for the doubters, I throw in this fact. Let's say that you believe they would not risk putting all of the experimental questions on one block. Here are the assumptions that you have to overcome to have a competitive advantage on this exam.

1. You must assume there are questions that are not graded on Step 1.
2. You must assume that all of the questions are all in one block.
3. You must somehow figure out which block is experimental.
4. You must risk testing your hypothesis by not "trying" on the experimental
block, even if you figure out which one it is.

Notice all 4 of these premises makes it almost impossible for one to have a competitive advantage on this exam. Therefore, I believe the NBME will never admit that it does have any experimental questions. Even so, you still have to overcome 2-4 to have an advantage. Thus, I believe that Step 1 is just like the GRE. It has one experimental section that does not count.

Do I have proof of this? No. Can I be flat out wrong? Yes. Still, I stand by my hypothesis, and I will dig in the annals of academic medicine journals until I figure out the truth. Just like I figured out which verbal section on the MCAT was experimental.

It is most like to be dispersed, otherwise ppl would just sleep through the block that wasn't going to count.

 

[Idiopathic]

Goljan told us during board review that there likely are no 'experimental' questions on the USMLE, because once they went computerized, they had all the questions they wouild need. He did say, however, that up to 20%(!) of the COMLEX is 'experimental'. I didnt believe either statement, by the way, but it was interesting. He was the first person I ever heard assert that there were no such questions on USMLE.

 

[fernj1975]

Well, maybe I might be suffering from selective amnesia but I don't remember seeing too many questions on the USMLE covering topics that were totally foreign to me (I remember one question specifically). If there are truly experimental questions on the USMLE then they must cover material that is currently not taught or slowing being implemented (e.g., molecular biology); thus I'm not fully convinced that there is a whole block of experimental questions, if there are some they might be a few mixed in between the blocks. This is totally anecdotal but I'm wondering if there are others who had a similar experience.

 

[Idiopathic]

The material wont be experimental, but new questions have to be 'ranked' based on difficulty level and also have to be tested on real students, to help determine if they are well-written, too hard, too easy, etc. The only practical way to do this is in a real-life test situation. So, there are questions in each test that are not scored, because the testers want to see how well they test, for a few years, after which they will be inserted into the test. It is simply quality control.

 

[fernj1975]

The material wont be experimental, but new questions have to be 'ranked' based on difficulty level and also have to be tested on real students, to help determine if they are well-written, too hard, too easy, etc. The only practical way to do this is in a real-life test situation. So, there are questions in each test that are not scored, because the testers want to see how well they test, for a few years, after which they will be inserted into the test. It is simply quality control.

So if you are right then It will be impossible to distinguish between the experimental and actual test questions since they cover the same material. This would explain my experience.

 

[Idiopathic]

I am right and the only way that you would have an idea is if they overload you on a specific subject (i.e. 50 thyroid questions) or something like that. More than likely, they have a pretty well-thought out way of accomplishing their goal...I have always asserted that we will not have any solid way of knowing which questions are real and which are experimental.

 

[dodo2]

I think they discard the questions that everybody gets right. They also discard questions that most get wrong. 😎