Red Line Challenge
Students receive an appellate brief excerpt that looks professionally drafted but contains multiple errors typical of AI-generated legal work: fabricated citations, citation drift, doctrinal distortion, assumed facts, and tone mismatch. Their task is to identify, classify, and log as many errors as possible before time runs out.
Purpose
This exercise builds the habit of treating verification as a professional act, not a chore at the end. Students receive an appellate brief excerpt that appears facially plausible but contains fabricated citations, citation drift, doctrinal distortion, assumed facts, and tone mismatch — errors associated with AI-generated legal work. Their job is not to correct the brief. It is to find problems, name them, and document their findings.
The exercise is intentionally difficult. Students will not catch everything in the allotted time, and that is by design. The point is for students to experience the verification burden directly: how quickly a polished, confident-sounding document can become dangerous when authority, facts, and standards have not been independently checked. Pattern recognition for AI error types — fabrication, drift, distortion, assumed fact, tone — only develops through repeated exposure.
The exercise also builds professional empathy. Students should understand not just how to verify but why verification is hard under real workload conditions. An attorney under deadline, reviewing what looks like a well-researched draft, faces the same pressure students experience during this exercise. That experience is part of what the debrief makes explicit.
Learning Goals
By the end of this exercise, students should be able to:
- Identify the range of errors that appear in AI-generated legal documents: fabricated citations, citation drift, doctrinal distortion, assumed facts, and tone mismatch.
- Explain why a real citation is not a verified citation, and why citation drift can be harder to catch than outright fabrication.
- Use a legal research platform to verify legal propositions, quotations, parentheticals, and case citations against primary sources.
- Produce a verification log documenting each finding in a format a supervisor could audit.
Materials
- Student brief: the appellate brief excerpt distributed to students. Includes only the Introduction and Argument sections. Does not include the instructor key, planted error list, or error count.
- Student worksheet: the error log students use to record suspected errors, classify issue types, note their verification strategy, and reflect on what they did and did not have time to check.
- Access to Westlaw, Lexis, or another legal research platform for manual verification. Students should not use automated brief-checking tools such as BriefCheck, BriefCatch, or Drafting Assistant — this is a manual verification exercise.
- Instructor packet (provided): includes the full error key, debrief structure, scoring guidance, and preparation checklist. Distribute to instructors only.
Setup
Distribute the student brief without telling students how many errors are planted, what types of errors appear, or which sections to focus on. Students should approach it as a practicing lawyer would: a document that looks professionally drafted and that a supervising attorney intends to rely on. Their job is to find problems, not produce a corrected version.
The exercise works best after students have been introduced to the error categories — citation fabrication, citation drift, doctrinal distortion, assumed facts, and tone mismatch — either through a prior class discussion or by distributing the category definitions at the start of the exercise. Providing the categories but not the counts or locations is the recommended setup for a first run.
Run of Show
Total, approximately 30–45 minutes in class; debrief in a later session
- Frame the task Distribute the brief without telling students it contains errors. Explain that they are acting as an associate who must verify the document before it goes to a supervising attorney or a court. They will not be rewriting anything — they are hunting for problems and logging what they find.
- Error hunt and log Students read the brief, mark every citation and assertion that depends on a citation, and attempt to verify each one. For each item, they log: what the brief claimed, what source they used to check it, and what they found — confirmed, miscited, misstated, or fabricated. The verification and logging happen simultaneously. Students will not catch everything. That is intentional.
- Submit and initial share Students submit their verification log. One or two findings are briefly shared with the class as a preview before the full debrief.
- Later class debrief In a later class, reveal the full error list. Discuss what students caught, what they missed, and what categories of error were hardest to spot. Individual written feedback can be substituted if a class debrief is not possible.
Student Instructions
You have received an appellate brief excerpt. A supervising attorney will rely on it. Your job is not to rewrite it — it is to find problems, classify them, and document what you find.
Do not assume the brief is correct. Do not assume it is wrong. Treat every citation, quotation, and assertion that depends on authority as unverified until you have checked it against the source itself.
Work in two passes:
- Triage: read once and mark every citation and every assertion that rests on a citation.
- Verify and log: attempt to confirm or refute each marked item using your legal research platform. Do not rely on the brief's own description of the sources. For each item, log what the brief claimed, what source you used to check it, and what you found — confirmed, miscited, misstated, or fabricated.
Use these categories when classifying errors:
- Citation Fabrication — the authority does not exist.
- Citation Drift — the authority exists but the brief misstates the holding, quotation, or standard.
- Doctrinal Distortion — the legal standard exists but is attributed to the wrong jurisdiction, court, or doctrine.
- Assumed Fact — the brief includes factual detail not supported by the record.
- Tone Mismatch — the language is inconsistent with professional legal advocacy.
Submit your verification log. Come to the debrief ready to say what you caught, what you missed, and how long it actually took.
Instructor Notes
Students will not catch every seeded error. That is the lesson. The exercise is intentionally hard — the point is not to find everything but to experience how much can slip past a competent reader under time pressure. Reveal the full error list only at the debrief, after students have submitted their logs. Do not tell students how many errors are planted before or during the exercise.
The most productive debriefs start with process, not answers. Ask students what they checked first — case existence, quotations, parentheticals, or legal standards — and why. Then move to what was hardest to catch. Students often focus on fabricated citations because they are the most visible form of hallucination. Use the debrief to emphasize that citation drift can be more dangerous: the case exists, the parenthetical sounds plausible, but the brief misstates the holding, inverts the standard, or attributes language to the wrong rule. A real citation creates false confidence in ways a fabricated citation does not.
Close the debrief by connecting findings to professional responsibility: competence (knowing how to use and verify AI-assisted work), candor (court submissions may not contain false authority), supervision (reviewing work prepared by AI tools or junior colleagues), and professional judgment (deciding when a document is not ready to move forward). The exercise should leave students with one working principle: a polished brief is not a verified brief. Individual written feedback on each student's log can be substituted if a class debrief is not possible.
Adaptation
Here are some suggestions on how to adapt this in other teaching contexts. The adaptations below have been limited to firm trainings and asynchronous suggestions, but there are other possible adaptations for workshops, seminars, and other contexts. Be creative!