Unofficial guide · not affiliated with the NCBE, the ABA, or any state bar ← SI Bar Exam · Exam Admin · Academy

Eli Item-Writing Academy

Academy › Item Craft

Flaw Clinic: The Dozen Defects That Kill MBE Items

About 45 minutes · Academy module: Flaw Clinic: The Dozen Defects That Kill MBE Items.

Learning goals

  • Recognize the twelve most common MBE item flaws on sight.
  • Diagnose cueing, unfocused stems, and non-parallel options in your own drafts.
  • Rewrite flawed items so the key is defensible to two independent cold solvers.

Lesson 1 — Cueing: The Item Answers Itself

Cueing is any feature that lets the examinee pick the key without knowing the doctrine: the longest option, the only option with a "because" clause, the option that repeats a distinctive phrase from the stem, or grammatical agreement (the stem's verb tense matching only one option). The fix is mechanical — make options parallel in length, structure, and phrasing — but the diagnosis requires reading your item like an adversary.

Lesson 2 — The Unfocused Stem And The Double-Barreled Call

An unfocused stem has no real call ("Which of the following is correct?"). A double-barreled call asks two things ("Did the court have jurisdiction, and was venue proper?"). Both defects make the item unanswerable as written: the examinee cannot know which decision earns the point. Fix by writing one call that names one decision.

Lesson 3 — Non-Parallel And Implausible Options

Options must be the same kind of phrase at the same level of specificity. "Yes, because the mailbox rule applies" alongside "No" and "The statute of limitations" is not an option set — it's a key with decorations. And every distractor must be genuinely choosable: an option no prepared examinee would pick doesn't discriminate, it just fills space.

Lesson 4 — The Flawed Key: Two Defensible Answers

The deadliest flaw is a key that isn't uniquely best. It usually comes from an unstated tested point (see Foundations) or from a fact pattern that supports two holdings. The standard: two independent solvers, solving cold, must converge on the key — and must be able to say why each distractor fails. If they can't, the item doesn't ship.

Lesson 5 — The Remaining Eight, Rapid-Fire

Absolute terms ("always," "never") in distractors make them trivially excludable. Negative calls ("Which of the following is NOT…") are legal but overused — they test careful reading more than doctrine. "All of the above" rewards partial knowledge and should never appear. Window dressing — facts that don't affect the answer — punishes careful readers. Trick facts that change the answer on a technicality the call didn't flag test reading, not law. Overlong stems (see the Fact-Pattern Workshop) bury the tested point. Culturally loaded scenarios belong in the Bias & Sensitivity module. Echoing the key's language in the stem ("the negligent driver… which theory applies?") gives the answer away.

Spot-the-flaw drill

(intended key: B)

Fact pattern: Police stop a driver for speeding. Without Miranda warnings, an officer asks, "Do you have anything illegal in the car?" The driver says, "Just a little marijuana." The driver moves to suppress.

A. Grant B. Deny, because a routine traffic stop is not Miranda custody, and Miranda warnings are therefore not required before the roadside questioning that occurred here C. Grant, because warnings are required D. Deny

Flaw: cueing — option B is conspicuously longer and more detailed, and options A/D are

cueing — option B is conspicuously longer and more detailed, and options A/D are bare conclusions while B/C carry reasoning.

Correction

The length and detail of B signal the key to any test-wise examinee. The rewrite makes all four options parallel "holdings with reasons" of similar length, so the examinee must know the custody doctrine to choose.

Model rewrite

— cold-solve verified: 2/2 (solvers chose B, B)

Fact pattern: Police stop a driver for speeding. While writing the ticket at the roadside, an officer asks, without giving Miranda warnings, "Do you have anything illegal in the car?" The driver answers, "Just a little marijuana." The driver moves to suppress the statement. How should the court rule?

A. Grant the motion, because all police questioning requires Miranda warnings B. Deny the motion, because a routine traffic stop is not Miranda custody C. Grant the motion, because the question was designed to elicit an incriminating response D. Deny the motion, because traffic stops are civil matters outside Miranda

Key: B

Drill key

Miranda requires custody + interrogation; a routine roadside stop is a brief detention, not custody. Both solvers converged on B and eliminated each distractor on doctrinal grounds.

Sources

  • Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–334.
  • Case, S. M., & Swanson, D. B. (2002). Constructing written test questions for the basic and clinical sciences (3rd ed.). National Board of Medical Examiners. (The classic flaw taxonomy — cueing, unfocused stems, "all of the above" — adapted here to MBE items.)

Finished this module?

Recording it adds the module to your Academy completion record on this device — module, track, level, and date, ready to download from your account page for faculty-development documentation.