Do Preprints and Open-Access Papers Get Cited by AI?

First question: can anything reach them?

Before asking whether an assistant cites academic sources, it is worth asking whether it can fetch them. Every discussion of AI and scholarship we have read skips this step, and it turns out to be the part with hard evidence in it, because robots.txt is a public file that anyone can read and nobody can spin.

So we read seventeen of them — preprint servers, a national archive, aggregators, institutional repositories and two commercial publishers — and checked each against seventeen named AI user agents, from GPTBot and OAI-SearchBot through ClaudeBot, PerplexityBot, CCBot, Google-Extended, Applebot-Extended, Bytespider and meta-externalagent.

Three things in that audit are worth stating flatly.

The open servers are open by default rather than by decision. arXiv's robots.txt opens with a comment — “Indiscriminate automated downloads from this site are not permitted” — and then, in the * group, explicitly allows /abs, /pdf and /html with a fifteen-second crawl delay. The only agent given a whole-site block anywhere in the file is SemrushBot, an SEO crawler. No AI agent is named, which means arXiv has not made a decision about AI crawling so much as declined to make one, and the general permission stands. PubMed Central is the mirror image in form and the same in effect: its file is Disallow: / with a list of explicit allowances, and /articles/ is one of them.

SSRN's block is narrow and pointed. Its entire robots.txt is four groups. Three of them are Disallow: / for GPTBot, ChatGPT-User and Google-Extended. The fourth is the general group, which disallows only /admin/, /tasks/ and /config/. So SSRN has blocked OpenAI's training and user-fetch agents and Google's AI-training opt-out token, and has not blocked ClaudeBot, PerplexityBot, CCBot, Amazonbot or anything else we probed. Crossref's API records SSRN's DOI prefix, 10.2139, as belonging to Elsevier BV, so this is a commercial publisher's selective posture rather than a scholarly principle. Which specific agents a publisher names, and why the lists differ so much between them, is the subject of which AI crawlers publishers name.

The reviewed version is often the blocked one. Springer Nature blocks thirteen of our seventeen agents on nature.com and on link.springer.com. A paper that exists twice — as an arXiv preprint and as a peer-reviewed Nature or Springer article — is reachable in its unreviewed form and unreachable in its reviewed one. Whatever an assistant ends up quoting, the version-of-record is the one with the wall around it. That is the same dynamic we measured among news publishers in whether news sites block AI crawlers, arriving in scholarship by a different route.

One caution on all of this. robots.txt is a request, and blocking a crawler is not the same as removing yourself from a system that has already ingested you, a distinction we set out in whether blocking a crawler removes you from search and in whether you can tell an AI not to train on your site. Much scholarly text also reaches model builders through intermediaries rather than direct crawls, including the corpus described in what Common Crawl is.

There is also a layer robots.txt does not express. PMC's own terms are more restrictive than its robots file: “The PMC Cloud Service, PMC OAI-PMH Service, E-Utilities and BioC API are the only services that may be used for automated retrieval of PMC content. Systematic retrieval (or bulk retrieval) of articles through any other automated process is prohibited.” A crawler reading only robots.txt would conclude that /articles/ is fair game at scale. The terms say otherwise.

What the citation evidence actually shows

Here the honest answer is short: almost nothing, and we are not going to inflate it.

The best independent measurement of AI citation behaviour we have found is the February 2026 evaluation by Suzgun and colleagues — six commercial chatbots, 2,100 questions derived from same-day news reporting, 12,600 model-question instances over fourteen days — which we analyse in which sources AI assistants actually cite. It finds that English Wikipedia is the single most cited source globally and that retrieval failures rather than reasoning failures account for more than 70% of errors. It reports no academic or preprint domain among the most-cited sources, and it was not designed to look for one: the questions were news questions, so the study tells us about news retrieval and is silent on scholarship. It is also a preprint, which is a fact we state every time we cite it and which is not incidental to this guide's subject.

Nothing on the operator side fills the gap. Google publishes an eligibility rule for its AI features — a page must be indexed and eligible to be shown in Search with a snippet — and does not publish how it selects among eligible pages. OpenAI publishes its crawlers and how to control them, and not how sources are chosen. Microsoft shows publishers their own citation counts while warning the numbers say nothing about “ranking, authority, or the role of any page within an individual answer”. We set that all out in what signals answer engines use to judge a source. None of it distinguishes a journal article from a preprint, because none of it describes source selection at all.

So the position is: we can measure reachability, we can measure what identifiers exist, and we cannot measure academic citation share. Anyone who tells you that a given share of AI citations goes to arXiv or PubMed is either quoting a vendor tool built on its own sampling or making it up. The errors documented in how often AI search tools cite the wrong source should also temper any assumption that a citation to a paper means the paper was read correctly.

What peer review does and does not guarantee

The servers themselves are the clearest authorities on this, and they are admirably blunt.

arXiv states that “the arXiv moderation process is not a peer-review process” and that “arXiv staff and moderators cannot give feedback on the submission”. All submissions are moderated; material is “expected to be self-contained and of interest, relevance, and value to the disciplines we serve”; arXiv “reserves the right to decline or remove any content”. What that buys is a filter against the obviously out-of-scope, not an assessment of whether the work is right.

medRxiv, operated by the non-profit openRxiv, is more explicit still, because the stakes in clinical literature are higher. It describes itself as “a free online archive and distribution server for complete but unpublished manuscripts (preprints) in the medical, clinical, and related health sciences”. Screening consists of “a basic screening process for offensive and/or non-scientific content and for material that might pose a health risk” plus a plagiarism check. And every preprint carries the warning: preprints “are preliminary reports of work that have not been certified by peer review” and “should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information”.

Read that last clause again if you are considering a press release built on a preprint. The server hosting the document asks journalists not to report it as established.

What peer review does guarantee is narrower than most non-academics assume, and it is worth being precise so that the preprint comparison is fair. Peer review establishes that named reviewers in the field read the manuscript and that an editor judged it publishable after their comments. It does not verify the underlying data, does not reproduce the analysis, and does not certify that the conclusions are true. It is a filter with a known and much-studied failure rate. The difference between a reviewed paper and a preprint is therefore a difference of degree — somebody independent looked versus nobody independent looked — rather than the difference between fact and speculation. But it is the only difference on offer, and it is the one an informed reader checks.

Whether any retrieval system checks it is a separate question, and the evidence says no. Nothing in any operator's published documentation mentions peer-review status as a factor. Meanwhile the reachability audit above shows the mechanical incentive running the other way, because the reviewed version is more likely to sit behind a publisher that blocks crawlers. If an assistant's behaviour were being designed around peer review, this is not what it would look like.

How a preprint differs from a published paper to a machine

Strip away the scholarly meaning and four practical differences remain.

Two documents, two identifiers. arXiv assigns its own: “All articles submitted to arXiv are automatically assigned DOIs that correspond to their arXiv ID”, under the prefix 10.48550, and arXiv also carries separate metadata fields “for linking to versions of an article outside arXiv”. So the same work can hold an arXiv DOI pointing at a versioned abstract page and a publisher DOI pointing at the version of record. A system that retrieves one has no obligation to know about the other, and a reader following a citation may land on either.

Versions. A preprint is mutable in a way a published paper is not: v1, v2, v3, with the arXiv DOI resolving to the current one. That is a feature for researchers and a hazard for anything quoting a number, because the number may have changed.

The label travels unevenly. PubMed Central is the interesting case. It is “a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health's National Library of Medicine”, and its holdings include formally published articles, author manuscripts deposited through the NIH Manuscript Submission System, and preprints from the NIH Preprint Pilot. A preprint in PMC sits in the same interface, with the same look and the same institutional halo, as a peer-reviewed article. Anyone — human or machine — treating “it's on the NIH site” as a proxy for review is making an error the archive itself does not invite but does make easy.

Free to read is not free to reuse. PMC “makes all content free to read (in some cases, following an embargo period)” but states that “free access does not mean that there is no copyright protection”. The Open Access Subset is the narrower collection of “journal articles and preprints that are made available under Creative Commons or similar licenses that allow more liberal redistribution and reuse”, and PMC warns that “not all articles in PMC are available for text mining and other reuse”. Open access, in the licence sense, is a property of individual articles rather than of the archive.

Do DOIs and Crossref metadata matter?

They matter, and not in the way they are usually sold. To see what a DOI is actually evidence of, look at what is in the registries.

Crossref's own position on access is generous and clearly stated: “No sign-up is required to use the REST API, and almost none of the metadata is subject to copyright, and you may use it for any purpose.” Its records cover journal articles, books, chapters, datasets, dissertations, grants, conference proceedings, peer reviews, reports, working papers, and preprints under the type posted-content. If you want the world's citation infrastructure to know your document exists, this is the door, and it is open.

Now the caveat, which is the whole point of the section. A DOI is a persistent identifier and nothing else. The DOI Foundation describes it as “a digital identifier of an object, any object — physical, digital, or abstract”, providing “a long-lasting reference to an object or to information about an object”, and is explicit that the persistence rests on organisational commitment rather than on technology. It is a promise by a registrant, not a verdict on the thing registered. The general lesson from whether identifier registries affect search visibility holds exactly here: identifiers are identity hygiene with a documented role in disambiguation, and are not a visibility product.

The registrant list makes the point better than any argument could.

The largest registrant of preprint DOIs in the world is a commercial publisher — Elsevier, through SSRN — and the seventh largest is a news organisation. Whatever posted-content means, it does not mean “scholarly work awaiting review”. It means a document type in a metadata schema, chosen by whoever registered it.

Then there is the fact most people get wrong. arXiv is not in Crossref. We confirmed it two ways: the DOI Foundation's registration-agency lookup for an arXiv DOI returns “DataCite”, and a Crossref API request for the prefix 10.48550 returns HTTP 404. DataCite holds 3,180,774 arXiv DOIs. Any tool, dashboard or analysis built on Crossref alone is blind to the largest preprint server in the physical sciences. That is not a criticism of Crossref, which does not claim otherwise; it is a warning about coverage claims made by people who query one registry and speak as if they had queried the literature.

What identifiers do buy is real and worth having. A DOI gives a document a stable address that survives a website redesign. Crossref or DataCite metadata puts the title, authors and date into an openly queryable graph. Google Scholar will index the document if the technical conditions are met: files “either in the HTML or in the PDF format”, PDFs with “searchable text”, every article “reachable from the homepage by following at most ten simple HTML links”, and metadata carrying the title, “the full name of at least the first author” and “the year of publication”. Those are the same conditions we set out for reviewed conference proceedings in whether conference talks build authority, and they apply identically to a preprint: Scholar's stated scope includes “journal papers, conference papers, technical reports, or their drafts, dissertations, pre-prints, post-prints, or abstracts”, and excludes “news or magazine articles, book reviews, and editorials”.

If your company wants to publish research of its own

The mechanics are easy and that is the problem. Anyone can post a PDF to a preprint server or an institutional repository, receive a DOI within days, and appear in Google Scholar shortly afterwards. None of those steps involves an independent person reading the work. A company that mistakes the resulting object for a peer-reviewed study will eventually be corrected by someone who checks, and the correction is worse than the original absence.

Four things are worth knowing before you do it.

A preprint is self-publication with an identifier. That is not an insult; it is the accurate description, and it is exactly what the servers say. arXiv's moderation “is not a peer-review process”. medRxiv's preprints “have not been certified by peer review”. If your document is good, this costs you nothing. If it is a marketing survey with a methods section, the label does not upgrade it and the permanence works against you.

Nobody independent will treat it as independent. Wikipedia's notability guideline for companies is the clearest published statement of the standard that gatekeepers apply. Coverage by “organization's personnel, owners, investors, (sub)contractors, vendors, distributors, suppliers, other business partners and associates, customers, competitors, sponsors and sponsorees” is not independent, and press releases, self-published material and “other works in which the company… talks about itself” are excluded outright. A company-authored preprint is squarely in that category, as is any study a company commissioned. We trace how that standard is applied in practice in how Wikipedia decides to delete an article.

Permanence is the underrated risk. A DOI is designed to keep resolving. A preprint's versions are retained. If your company publishes a claim about its market, its product or its competitors and the claim turns out to be wrong, you have created a durable, citable, machine-readable record of having made it. Marketing copy can be quietly edited; a registered preprint cannot. The whole value of the identifier system is that it stops things disappearing.

If you want the credibility, buy the review, not the venue. The only thing that converts a company document into evidence a third party will accept is a third party having examined it. The routes are ordinary and slow: co-author with academics who will insist on their own standards, submit to a journal or a reviewed conference and let it be rejected, commission an audit or a replication by a named independent party and publish the terms of reference, or release the data and the code so that anyone can check the analysis. Where none of that is possible, the honest presentation is the one that says so — label the work as company research, state the method and the sample, name who paid for it, and let it be judged as what it is. That is the same disclosure discipline that separates usable vendor research from unusable vendor research, and readers are better at spotting the difference than most communications plans assume.

The argument for doing it anyway is decent. A well-made, honestly labelled technical report with a DOI, real methods and released data is a better asset than a press release, it is citable, it is findable in Google Scholar, and on the evidence of our robots.txt audit it sits on infrastructure that AI crawlers are generally permitted to read. Just do not confuse having published with having been reviewed, and do not let anyone in your organisation describe the result as peer-reviewed. That single word, used wrongly, is the reputational failure this section exists to prevent.

Methodology and limitations

Three kinds of work sit behind this guide. The documentation review covers arXiv's moderation and DOI help pages, medRxiv's about page, PubMed Central's introduction and Open Access Subset pages, Crossref's REST API documentation, Google Scholar's inclusion guidelines and the DOI Foundation's definition of a DOI, each read on 26 September 2026 and quoted rather than paraphrased where the wording carries the meaning. The robots.txt audit fetched /robots.txt from seventeen hosts on 26 September 2026 with an identified research user agent, parsed each file into user-agent groups, and recorded which of seventeen named AI agents received a whole-site Disallow: /; the agent list and per-host results are given in the chart above and the audit is reproducible in a few minutes by anyone. The registry counts come from our own queries to the Crossref REST API and the DataCite API on the same date, with the exact filters stated in each chart source line. Seven limitations. First, robots.txt measures permission, not behaviour: it does not tell us whether any crawler obeys it or whether the content has already been ingested by other means. Second, it is a snapshot; publishers change these files frequently and Springer Nature's and SSRN's lists in particular should be re-read rather than trusted from this page. Third, three sources refused automated access and are therefore not quoted anywhere in this guide: SSRN's own FAQ returned HTTP 403, bioRxiv's about page returned HTTP 429 with a Cloudflare error 1015, and the Committee on Publication Ethics guidance on preprints returned HTTP 403. We have used SSRN's robots.txt and Crossref's record of its DOI prefix, which we did obtain, and we have said nothing about bioRxiv's own policies. Fourth, our claim that AI crawlers are permitted on the open servers rests on the absence of a named block, which is weaker evidence than an explicit permission and could change with one commit. Fifth, seventeen hosts is a purposive sample chosen to span preprint servers, a national archive, aggregators, institutional repositories and commercial publishers; it is not a random sample and the two institutional repositories in it disagree with each other sharply, so no general statement about repositories follows. Sixth, and most important, we found no published measurement of academic or preprint domain share in AI citation datasets that we could read in full, and the one independent citation evaluation we do rely on is a preprint that was designed around news questions and reports no academic domain among its most-cited sources. We therefore make no quantitative claim about how often assistants cite papers, and any such claim you encounter should be traced to its sampling method. Seventh, registry counts are live and rise continuously; treat every figure here as a reading taken on one day.

Conclusion

The reachability question has a clear answer and the citation question does not. Preprint servers and public archives are, with the exception of SSRN, open to AI crawlers because they have never named one; the commercial publishers holding the peer-reviewed versions of the same papers are frequently closed. That is an odd equilibrium, and it means the version of a finding an assistant can most easily reach is often the version nobody has checked. Whether assistants actually cite any of it at measurable rates is not something anyone can currently tell you, because no operator publishes source selection and the one independent citation dataset we have reviewed found no academic domain worth reporting. For a company, the practical reading is narrow and firm. A DOI, Crossref or DataCite metadata and a Google Scholar entry are cheap, legitimate and worth having for identity and findability, and they are not a substitute for anybody having read the work. A preprint is self-publication that happens to be permanent. If you publish research, label it honestly, state the method, name who paid, release what you can, and never let the word “peer-reviewed” near it unless a peer reviewed it.

Frequently Asked Questions

Sources

1. primary data Our own robots.txt audit. /robots.txt fetched from seventeen hosts on 26 September 2026 and parsed for whole-site blocks on seventeen named AI user agents. Files read directly, including arxiv.org/robots.txt, ssrn.com/robots.txt, pmc.ncbi.nlm.nih.gov/robots.txt, nature.com/robots.txt and eprints.lse.ac.uk/robots.txt.
2. primary data Crossref. Crossref REST API. Our own queries on 26 September 2026: total works; filter=type:journal-article; filter=type:posted-content; posted-date range for 2025; facet=publisher-name over posted content; prefixes/10.2139 returning owner “Elsevier BV”; prefixes/10.48550 returning HTTP 404.
3. primary data DataCite. DataCite REST API. Our own queries on 26 September 2026: total DOIs; count for prefix 10.48550; provider record identifying arXiv. Registration agency for a sample arXiv DOI verified through the DOI Foundation's registration-agency lookup, which returns “DataCite”.
4. policy document arXiv. arXiv moderation. “The arXiv moderation process is not a peer-review process”; moderators cannot give feedback; expectations of submitted material; right to decline or remove content. Read 26 September 2026.
5. policy document arXiv. arXiv DOIs. Automatic DOI assignment corresponding to the arXiv ID, the 10.48550 prefix, and continued support for external journal-reference and DOI fields. Read 26 September 2026.
6. policy document medRxiv (operated by openRxiv). About medRxiv. Definition, operator, screening for offensive or non-scientific content and health risk, plagiarism checking, and the disclaimer that preprints “have not been certified by peer review” and “should not be reported in news media as established information”. Read 26 September 2026.
7. policy document US National Library of Medicine. About PMC. Definition of PubMed Central, the three content routes including the NIH Preprint Pilot, and the statement that “free access does not mean that there is no copyright protection”. Read 26 September 2026.
8. policy document US National Library of Medicine. PMC Open Access Subset. Definition of the subset, the warning that “not all articles in PMC are available for text mining and other reuse”, and the restriction of automated retrieval to the PMC Cloud Service, OAI-PMH Service, E-Utilities and BioC API. Read 26 September 2026.
9. documentation Crossref. REST API documentation. “No sign-up is required to use the REST API, and almost none of the metadata is subject to copyright, and you may use it for any purpose”; record types including preprints as posted content; the polite pool. Read 26 September 2026.
10. documentation Google Scholar. Inclusion guidelines for webmasters. Scope including preprints and drafts, exclusion of news and magazine articles, and the technical requirements for format, searchable PDF text, link depth and metadata. Read 26 September 2026.
11. documentation DOI Foundation. What is a DOI? Definition of a DOI, its function as a long-lasting reference, and the statement that persistence rests on organisational commitment rather than technology. Read 26 September 2026.
12. guideline English Wikipedia. Wikipedia:Notability (organizations and companies). The list of related persons whose coverage is not independent, and the exclusion of press releases, self-published material and works in which a company talks about itself. Read 26 September 2026.
13. preprint Suzgun, M. et al., 2026. Evaluating Commercial AI Chatbots as News Intermediaries. arXiv preprint 2605.22785. Six commercial chatbots, 2,100 questions, 9–22 February 2026; English Wikipedia the most cited source globally; retrieval failures account for over 70% of errors. Preprint, not peer-reviewed; designed around news questions and reporting no academic domain among the most-cited sources. Read 26 September 2026.
14. access refused Three sources refused automated access and are quoted nowhere in this guide: SSRN's own FAQ at ssrn.com (HTTP 403), bioRxiv's about page at biorxiv.org (HTTP 429 with Cloudflare error 1015), and the Committee on Publication Ethics guidance on preprints at publicationethics.org (HTTP 403). Attempted 26 September 2026.