Hallucinated Citations: What They Are and Why They Look Real
A plain definition of hallucinated citations, the two properties every one of them shares, how they differ from miscited sources, and what the published audits measured.
Hallucinated citations are references that look like real scholarly sources but point to works that were never published. A chatbot or writing tool produces them when it generates text that has the shape of a citation without looking up a record. The author names, journal, year and DOI can all look right, and the paper still does not exist.
This guide is the definition and explainer. It covers what makes a citation hallucinated, how that differs from an ordinary citation error, and what the published audits found. For the step-by-step method of finding them in a draft, use our guide to spotting hallucinated citations in a thesis.
What is a hallucinated citation
A hallucinated citation is a reference to a work that has no record anywhere: no publisher page, no database entry, no library catalogue entry. The word "hallucination" comes from how researchers describe chatbot errors in general. Walters and Wilder (2023) open their study by noting that "factually incorrect responses (hallucinations) limit their utility".
The same study gives a usable test. It counts a citation as fabricated when the work has never been "published, presented, posted, or otherwise publicly disseminated". A citation counts as real when an actual work matches it closely on both title and authors.
That definition is strict on purpose. A real paper with the wrong year is not hallucinated. It is miscited, and the fix is different.
Which two properties collide in a hallucinated citation
A hallucinated citation is what you get when two properties meet in one reference. The first is a convincing form: it is built from real-looking parts in the right citation style. The second is a missing record: no matching work exists.
Either property alone is harmless or easy to catch. A badly formed reference with no record looks wrong at a glance. A convincing reference with a record is simply a citation. The trouble starts only when both are true at once, because the form hides the absence.
The audits show how convincing the form can be. In Walters and Wilder's data, most fabricated article, book and website citations named real journals, publishers or organisations. Among fabricated journal articles, only 2 percent named a journal that does not exist.
How hallucinated citations differ from other reference problems
Three different problems get called "fake citations", and each needs a different check. Sorting them first saves you from deleting a real source or keeping an invented one.
| Problem | What is wrong | Typical sign | Check and fix |
|---|---|---|---|
| Hallucinated citation | The work does not exist | Real journal, plausible authors, a DOI that fails or does not match | Search by DOI, title and author; if nothing exists, delete it |
| Miscited source | The work exists, details are wrong | Right title and authors, wrong year or pages | Compare each field with the publisher's record and correct it |
| Misused source | The work exists but does not say what you claim | A perfect reference on the wrong sentence | Read the passage, then rewrite the claim or change the source |
Only the first row is a hallucination. A reference that passes every registry check can still sit in the third row.
What a hallucinated citation is built from
A hallucinated citation is assembled field by field, and any field can be borrowed from reality. Here is a fill-in template showing where each part can come from. It is an illustration, not a real reference.
(Author A, Author B) (Year). (Title on the right topic). (Real journal name), (Volume)((Issue)), (Pages). https://doi.org/(string in the right format)
- Authors: names that sound right for the field, sometimes real researchers in that area.
- Title: a phrase that matches the topic of your sentence closely, which is what makes it feel relevant.
- Journal: a real journal in most cases, as the 2 percent figure above shows.
- Year, volume, issue and pages: numbers in a plausible range for that journal.
- DOI: a string in the right shape. The shape proves nothing, so open it and compare what loads with every field of the reference.
Because every field can look right, reading a reference closely does not tell you whether it exists. Only finding the record does.
How common hallucinated citations are in AI-generated text
The published audits found high rates, but the figures depend on the tool, the date and the definition used. Treat each number as a measurement of one study, not a fixed rate.
| Study | What it tested | What it found |
|---|---|---|
| Walters and Wilder (2023) | 636 citations in 84 short literature reviews written by GPT-3.5 and GPT-4 in April 2023 | 55% of GPT-3.5 and 18% of GPT-4 citations fabricated |
| Bhattacharyya et al. (2023) | 115 references in 30 medical papers written by ChatGPT-3.5 | 47% fabricated, 46% real but inaccurate, 7% real and accurate |
| Chelli et al. (2024) | References generated for 11 systematic reviews by GPT-3.5, GPT-4 and Bard | 39.6%, 28.6% and 91.4% hallucinated |
Chelli and colleagues counted a reference as hallucinated when any two of its title, first author and year were wrong. That is looser than Walters and Wilder's "never published" test, so the percentages are not directly comparable. Our explainer of the Walters and Wilder study sets out its method and every figure.
Why AI tools produce them
A chatbot writes text by predicting what comes next, and a citation is text with a very regular shape. Walters and Wilder describe ChatGPT as "a language-processing tool" rather than an information-processing tool. They suggest it may not recognise which parts of its text should be treated as bibliographic data.
In their data GPT-4 fabricated far less than GPT-3.5, yet still fabricated. The mechanism is covered in our guide on why AI makes up citations.
What hallucinated references mean for your thesis
A hallucinated reference cannot support any claim, because there is nothing behind it. Any reader, including an examiner, can paste a DOI into a resolver or search a title in a database. Walters and Wilder recommend that instructors examine the works cited when they screen work they suspect was written with a chatbot.
The practical rule is simple. A reference stays in your thesis only after you have found its record yourself and read the part you rely on. The per-reference method is in our guide to checking if a citation is real.
A first check on one reference
If you have one suspicious reference in front of you, run three quick tests. Each one tells you which row of the table above you are in.
- Paste the DOI after https://doi.org/ and see whether the page that opens has the same title and authors.
- Search the exact title in Crossref's metadata search, which searches the metadata of journal articles, books, datasets and more.
- If neither finds it, open the journal's own website and browse the volume and issue the reference names. Walters and Wilder used this as their final check.
If all three fail, treat the reference as hallucinated until a record turns up. If one of them finds a record with different details, you are in the miscited row instead.
What Thesisroom does with this
Thesisroom's Citation Check looks each reference up in eight registries: Crossref, OpenAlex, Semantic Scholar, PubMed, Retraction Watch, DOAJ, CORE and Unpaywall. It shows which references it could verify, which are retracted, and which need checking by hand. It never calls a reference fabricated, because a missing record can also mean a source registries do not hold.
The free citation checker takes a pasted reference list with no account needed, and uses a smaller set of registries. Neither one judges whether a real source supports your sentence. That reading stays with you.
Frequently asked questions
What are hallucinated references?
Hallucinated references are the same thing as hallucinated citations: entries in a reference list that point to works that were never published. They look complete and correctly styled, but no publisher page, database record or library catalogue entry exists for them.
Are hallucinated citations always from AI tools?
The term describes references produced by chatbots and other text generators. A person can also invent a reference, or misremember one badly enough that it matches nothing. Whatever the origin, the test is the same: if no record exists, the reference cannot stay.
Can I tell a hallucinated citation just by looking at it?
No. Every field can look right, and in Walters and Wilder's data most fabricated citations named real journals or publishers. The only reliable test is finding the record, by DOI, title search or the journal's own issue listing.
Does a working DOI prove a citation is real?
Not on its own. A working DOI only shows that some record exists under that string. Compare the title, authors, year and journal on the page that opens with your reference. If they differ, the citation is either miscited or hallucinated.
What percentage of AI-generated citations are hallucinated?
It depends on the study. Walters and Wilder found 55 percent for GPT-3.5 and 18 percent for GPT-4 in April 2023. Bhattacharyya and colleagues found 47 percent for ChatGPT-3.5 medical references. Definitions differ, so compare with care.
What should I do if I find one in my draft?
Delete it. Then decide whether the claim it supported still needs a source. If it does, search for a real paper that says it and read the relevant passage. If none exists, rewrite or cut the claim.
Sources
- Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13(1), Article 14045
- Bhattacharyya, M., Miller, V. M., Bhattacharyya, D., & Miller, L. E. (2023). High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus, 15(5)
- Chelli, M., et al. (2024). Hallucination rates and reference accuracy of ChatGPT and Bard for systematic reviews: Comparative analysis. Journal of Medical Internet Research, 26, e53164
- Crossref Metadata Search
How this guide was made: Thesisroom drafted this guide with the help of AI writing tools, then checked every fact, figure and source by hand against the references linked above.