Why AI Makes Up Citations and How to Catch Them

The short answer: the model is optimising for the wrong thing

Why does AI make up citations? A language model is not searching a database of academic papers and returning matches. It is generating the most statistically likely sequence of tokens given the context. A citation is a highly patterned piece of text, so the model has learned what a citation looks like: two or three names, a year, a title, a journal, a volume, a DOI. When asked for one, it generates a citation-shaped output.

If the model has actually seen the specific paper during training, and if the retrieval pathway is accurate, the citation may match a real one. If either condition fails, the model still produces a citation-shaped output, because the training signal rewards producing something that looks right rather than admitting the source does not exist. The output looks correct, sounds correct, and does not resolve when you check it.

What the audits actually show

The base rates for hallucinated citations are well documented and worth stating specifically.

Bhattacharyya and colleagues (2023), publishing in Cureus, audited 115 references generated by ChatGPT for medical research topics. Only 7% were fully authentic and accurate. Forty-seven percent were completely fabricated, and the remainder were partially wrong: real authors stitched to nonexistent papers, or real papers cited with wrong details.

Walters and Wilder (2023), publishing in Scientific Reports, examined 636 citations across 84 literature reviews. 55% of GPT-3.5 references were fabricated. GPT-4 fabricated 18% under the same protocol.

Chelli and colleagues (2024) tested systematic-review generation across multiple models. Hallucination rates ranged from 28.6% for the best model to 91.4% for the worst. DOI accuracy across the sample was 16 to 20%, meaning most generated DOIs do not resolve to the paper described.

At field scale, Topaz and colleagues (2026), publishing in The Lancet, examined 2.5 million PubMed-indexed papers. One in every 277 papers published in early 2026 contained at least one fabricated reference, up from one in every 2,828 in 2023. That is a roughly tenfold rise in three years, tracking the adoption of AI-assisted drafting in academic writing.

Why models produce fluent fictions

Three mechanisms converge to produce fabricated citations, and understanding them shapes the checks that catch fabrications.

The training objective rewards fluency, not truth. During training, a language model is scored on how well it predicts the next token given the previous ones. A model that produces plausible-sounding text gets rewarded even when the text is wrong. Truth is not a term in the loss function. This is why the outputs read as confident even when the underlying content is invented.

Citations are patterned enough to interpolate. Academic citations follow a tight format. The model has learned the pattern well: the names go here, the year goes there, the journal follows, and the DOI has a specific structure. When asked for a citation, the model can produce a syntactically correct one for almost any claim, drawing on the pattern rather than on a specific memory of a specific paper.

Retrieval is not built into most base models. A base language model does not have a live lookup to a citation database. It has a compressed representation of what it saw during training, and everything else is interpolation. Some tools bolt a retrieval step on top, which reduces but does not eliminate fabrication (Chelli et al., 2024). Base models without retrieval fabricate at the rates above.

The specific ways fabricated citations enter theses

Three routes account for most fabricated citations in doctoral drafts. Each has a specific catch.

Drafting: a candidate asks the tool to suggest references while writing a paragraph. The tool produces citation-shaped outputs, which the candidate copies into the bibliography. The catch is to resolve every DOI before the citation is saved to a reference manager.

Summarising: a candidate asks the tool to summarise a literature area. The summary includes citations that were never in any paper the model actually read. The catch is to treat the summary as a set of leads to verify against Crossref or OpenAlex, not as a set of confirmed sources.

Translating: a candidate writes a paragraph in one language and asks the tool to render it in another. The tool may add citations in the translation that were not in the original. The catch is to translate without citations, then add citations manually against the source list.

The checks that catch fabricated citations

Four checks catch most fabrications. Running them by hand for one citation is fast. Running them for a whole bibliography by hand is slow, which is how fabricated citations survive to submission.

• Resolve the DOI at https://doi.org/[DOI]. If it does not resolve, the citation is fabricated or wrong.
• Match the metadata: authors (all, in order), title, journal, year, volume, pages. Mismatches signal that the DOI has collided with a real paper by accident.
• Check for retraction status via Retraction Watch or the Crossref API.
• Verify claim-fit by opening the paper and finding the specific passage the citation is anchored to.

The first three are automatic and can run at bibliography scale. Thesisroom’s citation guard runs them in one pass, drawing on ten registries. The fourth is a reading task the writer has to do, but it is faster once the flagged sources are the ones that need attention.

What Thesisroom does with this

Thesisroom’s citation guard extracts every bibliography entry and verifies it deterministically. Fabricated citations fail the DOI check first. Real-but-miscited citations fail the metadata check. Retracted citations fail the Retraction Watch check. Each flag names the specific database check that produced it, so the response is targeted rather than guesswork.

The full method treats deterministic verification as the standard: AI where necessary, deterministic where possible. Citation verification is one of the places where deterministic checks catch what generative tools cannot police in themselves.

Frequently asked questions

Do newer models still hallucinate citations?

Yes, at lower rates than older ones. Walters and Wilder (2023) found GPT-4 fabricated 18% of citations, down from 55% for GPT-3.5. Chelli and colleagues (2024) found rates from 28.6% to 91.4% across models. Rates have fallen over time, but no current model reaches a level that makes verification optional.

Why does the DOI sometimes look valid but not resolve?

Because the model has learned the DOI format (a prefix, a slash, an alphanumeric suffix) and can generate strings that look valid without producing a string that is registered with Crossref. Fabricated DOIs pass a visual check and fail a resolution check, which is why the resolution check is the primary line of defence.

Does retrieval-augmented generation solve the problem?

It reduces the problem but does not eliminate it. Retrieval-augmented models look up sources during generation and are less likely to fabricate whole citations. They can still miscite the source they retrieved, or invent details that were not in the retrieval. Chelli and colleagues (2024) note that DOI accuracy remains a weak point even with retrieval.

What percentage of citations in AI-drafted papers are fabricated?

The range varies by model and topic. Bhattacharyya (2023) reported 47% completely fabricated in medical topics. Walters and Wilder (2023) reported 18% to 55% depending on model. Assume a meaningful share of any AI-assisted bibliography contains fabricated or miscited entries until verified.

Can I ask the AI to double-check its own citations?

Not reliably. The same model that produced the fabrication cannot detect the fabrication in general, because it has no ground-truth database to check against. Asking a different model with retrieval helps, but the deterministic check against Crossref is what actually catches fabrications.

What is the fastest way to check a bibliography?

Automate the DOI-resolve, metadata-match, and retraction checks. Tools that do this at bibliography scale save the hours a manual check would take. The Citation Guard runs the sweep against the academic record and flags anything that fails, with the reason for each flag.

One takeaway before you close this tab

Language models fabricate citations because they optimise for plausible text, and a citation is text. The check that catches fabrications is deterministic: does the DOI resolve, does the metadata match, has the paper been retracted, does it support the claim. Running the first three at bibliography scale is what Thesisroom’s citation guard does in one pass. The fourth stays with the writer, because reading is still reading.

References

Bhattacharyya, M., Miller, V. M., Bhattacharyya, D., & Miller, L. E. (2023). High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus, 15(5).

Chelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., et al. (2024). Hallucination rates and reference accuracy of ChatGPT and Bard for systematic reviews. Journal of Medical Internet Research, 26.

Topaz, M., et al. (2026). Analysis of fabricated references in PubMed-indexed literature. The Lancet.

Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13.