A polished answer can still be fiction.
This is not just a problem for lawyers. The same failure can enter a medical summary, financial analysis, insurance claim, family-history search, or business decision. Fluency is not evidence.
Use AI to accelerate the work. Never let it replace the check that makes the work trustworthy.
The New Mexico Supreme Court held attorney Stephen Aarons in contempt and fined him $5,000 after an AI-assisted appellate brief contained fabricated witnesses, fabricated testimony, and misrepresented legal authority.
The court order says Aarons admitted he used ChatGPT, did not verify the output, and did not tell his client about the false material.
The failure was not using AI. The failure was surrendering judgment to it.
AI does not retrieve truth simply because we ask it for a summary. It generates a plausible response from the material and instructions it receives. When records are incomplete, confusing, or too large, confidence can conceal invention.
The professional still owns the answer. Delegating the first draft does not delegate responsibility for the facts.
AI can organize records, identify themes, compare documents, and reveal where a human should look more closely. Used as an assistant, it can compress hours of mechanical review.
Its language can sound more certain than its evidence. That invites capable people to mistake polish for verification, especially when time is short and the answer confirms what they hoped to find.
- Require a link or page reference for every consequential factual claim.
- Open the original source. Do not verify an AI answer with another AI answer.
- Separate drafting from approval: the person signing, sending, or acting must check the evidence.
- If the source cannot be found, treat the claim as unverified—no matter how convincing it sounds.
Useful as an assistant. Dangerous as an authority.
Useful: 6/10. Dangerous: 8/10. AI can accelerate judgment. It cannot assume responsibility for it.