Distractor Engineering: Wrong Answers That Teach

Learning goals

Build distractors from the classic MBE error types: misread rule, right rule/wrong facts, near-miss doctrine, and overgeneralization.
Write a distractor rationale for every wrong option before the item ships.
Use distractor pull data (from Item Analysis) to retire dead options.

Lesson 1 — Distractors Are The Item

The key tests whether the examinee knows the doctrine. The distractors test whether they only sort of know it — and that is where discrimination lives. A distractor that attracts strong examinees is either a brilliant trap (good) or a second defensible answer (fatal). You cannot tell which without writing down, for each distractor, exactly which examinee error it captures.

Lesson 2 — The Four Error Families

1. The misread rule. The examinee knows the doctrine's name but not its elements: "negligence per se establishes strict liability" — wrong, it establishes breach. 2. Right rule, wrong facts. The doctrine is correctly stated but doesn't fit these facts: excited utterance offered for a statement made an hour later, calm and reflective. 3. The near-miss doctrine. A sibling rule that governs adjacent facts: general vs. specific jurisdiction, mailbox rule vs. authorized-means-of-acceptance. 4. The overgeneralization. A true statement of law that doesn't answer this call: "Congress has broad commerce power" on a question asking about the state's power.

Lesson 3 — One Error Per Distractor

Each distractor should capture exactly one error. A distractor that is wrong for two reasons is fine; a distractor whose error you cannot name is not a distractor — it's filler. Write the rationale first ("C captures the examinee who confuses revocation-effective-on-receipt with acceptance-effective-on-dispatch"), then the option text. If you can't write the rationale, you don't have a distractor yet.

Lesson 4 — Order And Balance

Rotate key position across your items — keys that cluster on B and C are a pattern examinees exploit. Keep the "because" clauses honest: a distractor's reason should be a plausible-sounding doctrinal statement, not word salad. And never make the key the outlier in length or specificity (see the Flaw Clinic's cueing lesson).

Lesson 5 — Reading The Pull Data

After administration, distractor analysis (see the Item Analysis module) shows who picked what. A distractor with near-zero pull is dead weight — replace it. A distractor that pulls top scorers harder than the key is a red flag for a miskey or a second defensible answer — pull the item for review. Healthy distractors pull the middle: chosen by some, rarely by the best.

Spot-the-flaw drill

(intended key: A)

Fact pattern: A city ordinance requires landlords to install smoke detectors in every rental unit. A landlord fails to install them. A fire injures a tenant; a working detector would have allowed escape. The tenant sues for negligence. What is the effect of the ordinance violation?

A. It establishes negligence per se, because the tenant is in the protected class and the injury is the type the ordinance aims to prevent B. It is irrelevant, because the moon is made of cheese C. It establishes strict liability for any fire in the building D. It is only evidence of negligence if the landlord knew about the ordinance

Flaw: distractor B is implausible filler — no prepared examinee would choose it, so th

distractor B is implausible filler — no prepared examinee would choose it, so the item is effectively three options, and the key's length cues it anyway.

Correction

Replace B with a genuine error-capture: the examinee who thinks negligence per se requires scienter is already covered by D, so B should capture a different error — the examinee who confuses the standard-of-care function of statutes with a damages rule. The rewrite below gives B the "irrelevant" error (a real examinee mistake: thinking ordinances can't set the standard of care).

Model rewrite

— cold-solve verified: 2/2 (solvers chose A, A)

Fact pattern: (same). What is the effect of the ordinance violation?

A. It establishes negligence per se, because the tenant is in the protected class and the injury is the type the ordinance aims to prevent B. It is irrelevant, because ordinances cannot set the standard of care C. It establishes strict liability for any fire in the building D. It is only evidence of negligence if the landlord knew about the ordinance

Key: A

Drill key

B captures the "statutes can't set tort duties" error; C captures the "negligence per se = strict liability" error; D captures the scienter error. All three are genuine examinee mistakes. Both solvers chose A.

Sources

Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309–334.
Attali, Y., & Fraenkel, T. (2000). The point-biserial as a discriminator index. Applied Psychological Measurement — on why distractor-level analysis matters beyond the key.