How to Catch a Fake AI Citation in 30 Seconds

Large language models sometimes generate a citation that looks perfect — author, journal, year, page — but the paper does not exist. A New York lawyer was famously sanctioned in 2023 for filing a brief with six ChatGPT-invented court cases. Every citation an AI gives you needs to be verified before you turn it in. The 30-second check: search the exact title in Google Scholar; search the DOI on doi.org; search the author + year. If all three return nothing, the citation is fake.

Some examples

• Mata v. Avianca (2023): a NY lawyer used ChatGPT, cited six fake cases, was fined $5,000 and made famous; the cases looked real down to the docket numbers.
• Studies have measured GPT-4's academic citation hallucination rate at 18-30% in 2023; it dropped with newer models but is not zero.
• Real DOIs always start with '10.' followed by numbers — try doi.org/[paste the DOI] and if it 404s, the paper doesn't exist.
• Tools like Consensus.app and Elicit only return real, indexed papers — they cannot hallucinate citations because they retrieve from a real database.

Try it!

Ask ChatGPT for 'three peer-reviewed studies on [your essay topic] with full citations.' Check each one in Google Scholar. Count how many turn out to be fake. Now you know your default fact-check rate.