The Basic Rules for One-Best-Answer Items
- Each item should focus on an important concept, typically a common or potentially catastrophic clinical problem. Dont waste testing time with questions assessing knowledge of trivial facts. Focus on problems that would be encountered in real life. Avoid trivial, tricky, or overly complex questions.
- Each item should assess application of knowledge, not recall of an isolated fact. The item stems may be relatively long; the options should be short. Clinical vignettes provide a good basis for a question. For the clinical sciences, each should begin with the presenting problem of a patient, followed by the history (including duration of signs and symptoms), physical findings, results of diagnostic studies, initial treatment, subsequent findings, etc. Vignettes may include only a subset of this information, but the information should be provided in this specified order. For the basic sciences, patient vignettes may be very brief; laboratory vignettes are also appropriate.
- The stem of the item must pose a clear question, and it should be possible to arrive at an answer with the options covered. To determine if the question is focused, cover up the options and see if the question is clear and if the examinees can pose an answer based only on the stem. Rewrite the stem and/or options if they could not.
- All distractors (ie, incorrect options) should be homogeneous. They should fall into the same category as the correct answer (eg, all diagnoses, tests, treatments, prognoses, disposition alternatives). Rewrite any dissimilar distractors. Avoid using double options (eg, do W and X; do Y because of Z) unless the correct answer and all distractors are double options. Rewrite double options to focus on a single point. All distractors should be plausible, grammatically consistent, logically compatible, and of the same (relative) length as the correct answer. Order the options in logical order (eg, numeric), or in alphabetical order.
- Avoid technical item flaws that provide special benefit to testwise examinees or that pose irrelevant difficulty.
Do
NOT write any questions of the form Which of the following statements is correct? or Each of the following statements is correct EXCEPT. These questions are unfocused and have heterogeneous options.
Subject each question to the five tests implied by the above rules. If a question passes all five, it is probably well-phrased and focused on an appropriate topic.