Question on Experimental

[starburstsbabe]

---

Members don't see this ad.

I hope this isn't a stupid question.. but am I right that experimental questions do not count? I heard for Comlex that almost 30% of the test is experimentals.. so your 3-digit score will not be affected by them.. correct? Also is this the same with the USMLE? 😕

 

[AndyRSC]

Per First Aid, up to 10-20% of the test questions are considered experimental and don't count towards your grade on the USMLE.

As far as COMLEX, I haven't seen a defined percentage. I can tell you that COMLEX had far less far-fetched or near-impossible questions than the USMLE.

 

[ijn]

Based on my exam yesterday I don't know if there really are experimental questions. There were some poorly written questions, but I felt everything was pretty standard fare. I think maybe people get flustered by questions where you're supposed to use general principles to answer something that is unfamiliar. It's fair game to be given a question on some random ass gene that no one has ever heard of before and when provided some basic info, then be expected to make extrapolations from that.

 

[zof881]

Based on my exam yesterday I don't know if there really are experimental questions. There were some poorly written questions, but I felt everything was pretty standard fare. I think maybe people get flustered by questions where you're supposed to use general principles to answer something that is unfamiliar. It's fair game to be given a question on some random ass gene that no one has ever heard of before and when provided some basic info, then be expected to make extrapolations from that.

ijn,

I felt the exact same way. I'd read reviews of people talking about how they "knew" such and such was an experimental question because it was on some arcane topic. The first one I got on something like panic started to set in while I read the question. When I got to the end however, I realized they were just using something really unfamiliar to actually test a familiar concept.

Similarly with people saying they found "mistake" questions, with arrows the wrong way etc. I didn't notice anything like that. One of my professors, who supposedly used to write USMLE questions, told me that there is a group of people (different from the actual question writers) who's sole job is to make sure questions make sense and are internally consistent. I suspect most people who think they've found a mistake on a question, really just didn't understand the question or concept they were testing.

 

[ijn]

There definitely are mistakes on the test. One of my questions had me calculate stroke volume and told me the answers were in units of [ml/min]. Obviously those are the units of cardiac output. Stroke volumes are errrhm... volumes per beat. Yeah. That kind of stuff slips through the cracks.

 

[Radonkulous]

There definitely are mistakes on the test. One of my questions had me calculate stroke volume and told me the answers were in units of [ml/min]. Obviously those are the units of cardiac output. Stroke volumes are errrhm... volumes per beat. Yeah. That kind of stuff slips through the cracks.

ijn, how did you feel coming out? Any guess on your score or number missed?

 

[ijn]

ijn, how did you feel coming out? Any guess on your score or number missed?

No clue honestly. I think I marked about 6 per block. I could wake up July 13th with a 240 or a 270. Likely somewhere in between. My 3 school CBSEs predicted a 275.

 

[ijn]

.

 

[AndyRSC]

I've gotten several that I'd consider as such. One that I remember vividly tested organic chemistry II. You had to know the name of a specific reaction that split a certain compound in a particular way. Since organic chemistry was a pre-requisite to medical school for nigh all of us, I somewhat agree that it was fair game. It's definitely not a "familiar concept" if you go by a typical med school curriculum, First Aid, etc.

Another question was so specific and so... well, experimental, that I still haven't found a definitive answer - and, believe me, I've looked. Several classmates remembered similar questions to the T; again, no general principles to apply to tackle them, no clues, no extrapolations to make; just had to know it.

 

[ijn]

I mean I definitely had questions that required detailed knowledge beyond any review book to the level that maybe even phloston would gloss over it as low yield. I'd say it's a bitch question and I was okay with missing it.

I did have a handful of questions where I still can't find the answer after exhaustive searching. Maybe I'm just looking in the wrong sources. I still wouldn't call that experimental. Maybe poorly written. Or it's a necessary evil to prevent the next Pollux from getting a literal 100%.

The whole process is so cryptic. Perhaps some questions aren't worth any points, but I think what the NBME really does is tests questions on their subject exams and then filters them into the STEP exam. I am 100% certain that questions from my end of year pathology and pharmacology NBME subject exams from April were on my June STEP 1 exam.

 

[Aclamity]

I've been told (by someone reliable) that sometimes NBME uses these experimental questions to gauge what they've been teaching in med school. Maybe those questions won't turn into actual questions later down the road, but it's the NBME's way of data mining to see approx what % of students have been exposed to that topic. Thus, I think it's safe to say that at least SOME of the arcane topics could be experimental.

That being said, experimental questions would also be useless to NBME if they can't use them in future exams, so I'm sure there are many many experimental questions out there that we wouldn't think twice about while taking the exam--they're just like any other question.

I also definitely found mistakes in question stems, and I'm pretty sure I was not just missing the big idea or overlooking something. I'm a little surprised that something so blatant made it through the screening process, but when you think about it these questions are so important to us that we actually read them extremely carefully and may be able to pick up stuff that the proofreaders missed.

 

[Dr McSexy]

Perhaps some questions aren't worth any points, but I think what the NBME really does is tests questions on their subject exams and then filters them into the STEP exam. I am 100% certain that questions from my end of year pathology and pharmacology NBME subject exams from April were on my June STEP 1 exam.

I was told by a professor who writes questions for the USMLE(and on some of the review committees) that the NBME shelf exams are made up of retired USMLE questions or those in the process of being retired. Dunno though.

 

[lrkoehle]

I think a lot of people are under the impression that really hard = experimental. That just can't be the case. It also doesn't make sense that most of the experimentals don't count. That's why we're all waiting until July 11th to hear on our scores, so they can aggregate the data on these new/experimental/whatever-you-want-to-call-them questions. As someone else said, those really hard questions are in there to help with the curve and prevent people from actually getting a 100%.

I also would be willing to bet they throw out a lot of questions for being too easy like they do for being too hard. If everyone gets a question correct, then it doesn't really differentiate between people either. Maybe all those questions we found shockingly easy were actually experimental and are going to get thrown out...😱

 

[Siggy]

I also would be willing to bet they throw out a lot of questions for being too easy like they do for being too hard. If everyone gets a question correct, then it doesn't really differentiate between people either.

I doubt this actually. The USMLE, in design, is a pass/fail examination used to determine if someone meets the minimum requirements for licensure. It's not designed to sort test takers like the SAT, ACT, and MCAT are. Residencies use it because it's pretty much the only standardized tool they have, not because it's designed for it.

 

[lrkoehle]

I doubt this actually. The USMLE, in design, is a pass/fail examination used to determine if someone meets the minimum requirements for licensure. It's not designed to sort test takers like the SAT, ACT, and MCAT are. Residencies use it because it's pretty much the only standardized tool they have, not because it's designed for it.

I agree with that as the original premise, but it does seem to have transformed into a test with a desire to have a nice normalized curve. If it was truly intended to be strictly a P/F exam, then why throw in those extremely difficult questions to throw people off? Why not just a good well-rounded exam that tests a base level knowledge of basic sciences amongst second year medical students? They clearly want a normalized curve, and so I do think they would probably get rid of ones that are too easy.

Of course I'm only speculating like the rest of us, because the NBME likes to keep everything they do shrouded in a cloud of secrecy...

 

[Valadi]

ijn,

I felt the exact same way. I'd read reviews of people talking about how they "knew" such and such was an experimental question because it was on some arcane topic. The first one I got on something like panic started to set in while I read the question. When I got to the end however, I realized they were just using something really unfamiliar to actually test a familiar concept.

Similarly with people saying they found "mistake" questions, with arrows the wrong way etc. I didn't notice anything like that. One of my professors, who supposedly used to write USMLE questions, told me that there is a group of people (different from the actual question writers) who's sole job is to make sure questions make sense and are internally consistent. I suspect most people who think they've found a mistake on a question, really just didn't understand the question or concept they were testing.

I agree. I also had a prof who wrote for them and he described it as "not even worth it" for the money because of how much supporting evidence you had to provide for each question/answer. These questions are pretty thoroughly scrutinized.

 

[Sunnyside7]

No clue honestly. I think I marked about 6 per block. I could wake up July 13th with a 240 or a 270. Likely somewhere in between. My 3 school CBSEs predicted a 275.

See i felt like you did on the test and was projected at a similar score. What did you end up with, im curious where mine will be as well.