Retraction marks near links to retracted papers on Wikipedia: an exploratory mechanical count in English and eight other...

Exploratory, descriptive note. The numbers below count where a fixed string rule did or did not find a retraction mark in the wikitext near a link to a retracted paper. They are not measurements of whether articles actually tell readers that a paper was retracted, and the rule has not been checked by human coders.

Summary

We matched DOI and PubMed links in one Wikipedia dump run (labelled 20261001) against the Retraction Watch database and counted (article, retracted paper) pairs in the article namespace. A fixed string rule then looked in a detection window around each link (the enclosing reference element, or the line, plus 300 characters on each side) for a retraction template, a retraction word, or a link to the retraction notice.

English Wikipedia: the rule found no mark for 59 of 508 located pairs (11.6%).
Eight purposively selected non-English editions (German, French, Spanish, Russian, Japanese, Chinese, Korean, Indonesian), pairs pooled: no mark found for 543 of 773 (70.2%); per edition from 56.3% (German) to 84.1% (Spanish).
A retraction template was recorded near the link for 82.9% (421/508) of English pairs and 4.4% (34/773) of pooled pairs in the eight editions (lower bounds; see "Detection elements").

These are mechanical counts on a selected set of pairs (only links written as doi.org or PubMed URLs). They describe one dump run. They do not show why the editions differ, whether readers were misled, or what share of retracted-paper citations in these editions actually lack a retraction note.

Background

A retracted paper should not normally be used as unqualified support for a claim, but citations already in place remain until someone edits them (citing a retracted paper to discuss the retraction itself is legitimate). For English Wikipedia, Shi et al. followed the history of 1,181 citations to retracted papers and reported that 71.6% were initially problematic and in need of reader-facing repair (added before the retraction, or after it without a proper warning), with a median persistence of 3.68 years; in their survival analysis, bot-mediated flagging by RetractionBot was among the factors associated with a higher hazard of correction (an association, not a causal estimate). RetractionBot runs only on English Wikipedia. Earlier, Nicholson et al. found 118 retracted papers cited in 297 English Wikipedia articles (March 2018 dump) and, in a random sample of 50 such articles, 25 (50%) did not acknowledge the retraction.

We did not run a systematic literature search, and we make no claim that this comparison across editions is new.

Data and method

Editions. English plus nine other editions chosen before counting to mix size, script and whether a retraction template is linked from the Wikidata item Q14442550 (linked for de, fr, ja, zh, ko, id; not for es, ru): German, French, Spanish, Russian, Japanese, Chinese, Korean, Indonesian and Swahili. The selection is purposive and does not represent non-English Wikipedia as a whole. Swahili had been examined in a pilot before the analysis plan was written (see Appendix A6); it yielded only 3 pairs and is reported as a case, not in the comparison.

Inputs. For each edition, the externallinks table, the pages-articles-multistream index and, by HTTP range requests, the wikitext of matched pages from the dump run labelled 20261001 (downloaded 2026-10-24). A dump run is not an exact single-day snapshot. Retraction list: Retraction Watch database as published by Crossref (CC0; citation requested), file version "Retraction Watch data for 2026-10-22", retrieved 2026-10-24.

Pairs. A pair is (article, retracted paper) where the article links to the paper's DOI via doi.org or to its PMID via PubMed. Only article-namespace pages are counted. We do not distinguish reference-list citations from links in body text or lists. A paper linked several times in one article is one pair. Citations that give neither a doi.org nor a PubMed link (publisher URLs, plain-text DOIs, no identifier) are not included. Items in Cochrane Database of Systematic Reviews were excluded from the main count by a rule set before counting, because many Cochrane "retractions" are version replacements; Appendix A5 shows every edition with them added back.

Detection rule. Around every occurrence of the paper's identifier in the wikitext, the window is the enclosing <ref> element (or the line) plus 300 characters on each side. A pair is "marked" if any window contains (i) a template whose name contains a retraction stem, (ii) a retraction word from a per-language list (the English stem "retract" counts in every edition, which also catches titles prefixed "RETRACTED:"), or (iii) the DOI or PMID of the retraction notice. Otherwise the rule "found no mark". The rule has the same form in every edition, but the word lists differ by language, while the same multilingual template-stem list is applied in every edition (Appendix A3); inflections and template aliases not on the lists are missed, and 300 characters hold different amounts of text in different scripts. The rule was not shown to have equal accuracy across editions.

Analysis plan. The definitions, the edition list and the stopping rules were written in an internal analysis plan before the main counts (dated in private version control, 2026-10-24, about four hours before the first results; not publicly registered, and not blind for Swahili). Every departure from it is listed in Appendix A6.

What the AI reading check can and cannot say

No human coded any window. During the analysis (2026-10-24), an AI model (Anthropic Claude) read the detection window text of a small random sample of located pairs in each edition. The reader saw the rule's output for each pair, so the reading was not blind. The exact model version, the prompt and the list of sampled pairs were not kept, and the split of each sample between rule-marked and rule-unmarked pairs is unknown, so we report no agreement rates, sensitivity or specificity. The check is reported only for the failure modes it exposed: the rule counted the name "Retraction Watch" inside an expression-of-concern template as a mark (French), counted a retraction word about a neighbouring paper as a mark (Korean), and missed a Swahili retraction word because the Swahili word list was empty (Appendix A6, item 5). The reader saw only the window, so a retraction note elsewhere in the article could not be found by this check either. It is weak evidence about string matching inside the window, and none about annotation status.

Results

Counts over all detected pairs in one fixed dump run. They are not sample estimates, and no intervals are given (Appendix A4).

English (en): Pairs: 514; Articles: 366; Located pairs: 508; Rule found no mark: 59 (11.6%); Template found: 421 (82.9%)
Spanish (es): Pairs: 126; Articles: 112; Located pairs: 126; Rule found no mark: 106 (84.1%); Template found: 0 (0.0%)
Chinese (zh): Pairs: 93; Articles: 88; Located pairs: 92; Rule found no mark: 76 (82.6%); Template found: 2 (2.2%)
Russian (ru): Pairs: 123; Articles: 111; Located pairs: 123; Rule found no mark: 96 (78.0%); Template found: 1 (0.8%)
Japanese (ja): Pairs: 160; Articles: 123; Located pairs: 160; Rule found no mark: 103 (64.4%); Template found: 11 (6.9%)
French (fr): Pairs: 130; Articles: 114; Located pairs: 128; Rule found no mark: 78 (60.9%); Template found: 3 (2.3%)
Indonesian (id): Pairs: 33; Articles: 30; Located pairs: 33; Rule found no mark: 20 (60.6%); Template found: 7 (21.2%)
Korean (ko): Pairs: 40; Articles: 38; Located pairs: 40; Rule found no mark: 24 (60.0%); Template found: 10 (25.0%)
German (de): Pairs: 72; Articles: 60; Located pairs: 71; Rule found no mark: 40 (56.3%); Template found: 0 (0.0%)
8 editions pooled: Pairs: 777; Articles: 676; Located pairs: 773; Rule found no mark: 543 (70.2%); Template found: 34 (4.4%)
Swahili (sw), case: Pairs: 3; Articles: 3; Located pairs: 3; Rule found no mark: 2; Template found: 0

"Located" means the identifier was found in the wikitext (unlocated pairs: en 6, de 1, fr 2, zh 1). Treating all unlocated pairs as marked or as unmarked gives 11.5% to 12.6% for English and 69.9% to 70.4% for the pooled eight. The pooled share weights editions by their number of pairs; the unweighted mean of the eight edition shares is 68.4%.

Dot plot of the share of located pairs where the string rule found no retraction mark: English 11.6%, the eight selected editions between 56.3% (German) and 84.1% (Spanish), pooled 70.2%

Figure: share of located (article, retracted paper) pairs where the string rule found no retraction mark in its detection window (the enclosing reference element or line, plus 300 characters on each side). Swahili (3 pairs) is not drawn.

Selection of pairs. Only citations carrying a doi.org or PubMed link enter the count. Kokash and Colavizza report that only about 5% of citations in English Wikipedia include DOIs, and that for 38% of the English citations they looked up in citation databases an identifier could be added. How often citations to retracted papers lack such links, and whether this differs by edition, was not measured here. If it differs, the comparison between editions is biased in an unknown direction.

Detection elements. The marked or unmarked outcome uses every occurrence window of a pair, but the element flags were recorded from one window only: the first occurrence window (in text order) that contained any element. A template or notice link in a later window of the same pair is not recorded, so the element counts below, including 421/508 and 34/773 for templates, are lower bounds; the "no mark found" column is exact under the rule. Within a window the elements overlap: every template recorded also matched the word element, because all template names on the list contain a retraction stem. The table assigns each located pair to one category only, in the order template, then word, then notice link. It describes what the rule matched, not how much of the difference between editions any element accounts for.

en: Template (with or without other elements): 421; Word, no template: 28; Notice link only: 0; No mark found: 59; Located: 508
de: Template (with or without other elements): 0; Word, no template: 27; Notice link only: 4; No mark found: 40; Located: 71
fr: Template (with or without other elements): 3; Word, no template: 47; Notice link only: 0; No mark found: 78; Located: 128
es: Template (with or without other elements): 0; Word, no template: 19; Notice link only: 1; No mark found: 106; Located: 126
ru: Template (with or without other elements): 1; Word, no template: 26; Notice link only: 0; No mark found: 96; Located: 123
ja: Template (with or without other elements): 11; Word, no template: 45; Notice link only: 1; No mark found: 103; Located: 160
zh: Template (with or without other elements): 2; Word, no template: 14; Notice link only: 0; No mark found: 76; Located: 92
ko: Template (with or without other elements): 10; Word, no template: 6; Notice link only: 0; No mark found: 24; Located: 40
id: Template (with or without other elements): 7; Word, no template: 6; Notice link only: 0; No mark found: 20; Located: 33
8 pooled: Template (with or without other elements): 34; Word, no template: 190; Notice link only: 6; No mark found: 543; Located: 773

German and French have a retraction template linked from Wikidata Q14442550, but it was recorded for 0 and 3 pairs respectively. Outside English, most marks the rule found were words: the "RETRACTED:" title prefix or a sentence describing the retraction.

Concentration. In every edition the median retracted paper appears in one article, and no single paper accounts for more than 10% of an edition's no-mark pairs.

en: Unique papers: 378; Articles per paper, median (max): 1 (18); Largest single paper's share of no-mark pairs: 8.5% (5/59)
de: Unique papers: 64; Articles per paper, median (max): 1 (3); Largest single paper's share of no-mark pairs: 5.0% (2/40)
fr: Unique papers: 100; Articles per paper, median (max): 1 (6); Largest single paper's share of no-mark pairs: 3.8% (3/78)
es: Unique papers: 88; Articles per paper, median (max): 1 (10); Largest single paper's share of no-mark pairs: 9.4% (10/106)
ru: Unique papers: 90; Articles per paper, median (max): 1 (5); Largest single paper's share of no-mark pairs: 5.2% (5/96)
ja: Unique papers: 124; Articles per paper, median (max): 1 (6); Largest single paper's share of no-mark pairs: 5.8% (6/103)
zh: Unique papers: 80; Articles per paper, median (max): 1 (3); Largest single paper's share of no-mark pairs: 3.9% (3/76)
ko: Unique papers: 39; Articles per paper, median (max): 1 (2); Largest single paper's share of no-mark pairs: 8.3% (2/24)
id: Unique papers: 30; Articles per paper, median (max): 1 (3); Largest single paper's share of no-mark pairs: 10.0% (2/20)

Across the eight editions, 396 unique papers form the 777 pairs; the most-linked paper (Wakefield et al., Lancet 1998) forms 21 pairs, 2 of them with no mark found (0.4% of the 543). The largest no-mark count for any one paper is 15 (2.8% of the 543), reached by two other papers. The ten papers with the most no-mark pairs account for 17.3% of them. Dropping any single paper moves the pooled share between 69.7% and 71.9%. Giving each paper equal weight (the mean, over papers, of the no-mark share among its located pairs) gives 71.1% for the pooled eight and 12.9% for English. At the article level, 504 of the 672 articles in the eight editions with at least one located pair (75.0%) have at least one pair where no mark was found; four further articles have no located pair.

What this note does not show

Not annotation status. The count is of pairs where the rule found nothing in the window. A note further away in the article, in another section, or in words not on the list counts as "no mark"; a retraction sentence about a neighbouring paper counts as a mark. No human validated the rule in any edition, and its error rates may differ by edition, so part of the difference between editions may come from the rule.
Not a cause. The bot exists only on English Wikipedia, so its presence cannot be separated from every other difference between English and the other editions (editor numbers, subject mix, citation habits).
Not reader harm. We did not measure whether readers were misled.
Not all citations. Only doi.org and PubMed links; see "Selection of pairs" above.
Not timing. We do not know whether each citation was added before or after the retraction.
Not all retractions. Cochrane items are excluded from the main count; adding all of them back changes the pooled eight from 70.2% to 71.9% (Appendix A5).
Not non-English Wikipedia in general. Eight purposively chosen editions.

Disclosure

The text was drafted with AI assistance (Anthropic Claude) and checked against the cited sources and the replication files. The AI reading check described above was done by Claude. The note was reviewed by an AI reviewer (OpenAI Codex). No Glasp user data was used. No external funding. No known conflicts of interest.

Replication

Replication files (code and data): deposited with this note. https://doi.org/10.5281/zenodo.23838830

The package contains the scripts (download, extraction, page fetch, retraction list, detection, revision, the full F4 check, the statistics added for this note, and the figure) and aggregate outputs: the per-edition table, the summary with the sensitivity values, the statistics added for this note (stage counts, element decomposition, concentration, Cochrane re-inclusion), and the per-edition F4 results. The pair-level file (one row per article and retracted paper, with the Wikipedia page ID) is not published, so no per-article list of pages without a mark is released. The dumps and the Retraction Watch CSV are not redistributed; they are public (see Sources). Every primary outcome count, sensitivity value, concentration figure and final F4 figure in this note is in the deposited outputs, but they were computed from the pair-level file, so they cannot be recomputed from the deposit alone. The run-record figures identified in A4, A6 and A7 (the English non-article pairs, the superseded F4 values, the pilot and download facts) and the AI reading check cannot be recomputed either. Regenerating the pair-level file needs the dumps, and exact regeneration is limited because checksums of the downloaded copies were not recorded (Appendix A7).

Sources

Wikimedia dumps, run 20261001, files {wiki}-20261001-externallinks.sql.gz, {wiki}-20261001-pages-articles-multistream-index.txt.bz2 and byte ranges of {wiki}-20261001-pages-articles-multistream.xml.bz2 for en, de, fr, es, ru, ja, zh, ko, id, sw; retrieved 2026-10-24. Checksums for each file are published by Wikimedia in {wiki}-20261001-sha1sums.txt in the same directory. This note reproduces no article text, only aggregated facts. https://dumps.wikimedia.org/
Retraction Watch Database, published by Crossref. retraction_watch.csv, version "Retraction Watch data for 2026-10-22" (GitLab commit 13d54397b963e8eba71e463243de00a88e441f9a, committed 2026-10-22 23:00 UTC), retrieved 05:36 JST on 2026-10-24 (20:36 UTC on 2026-10-23, before the next daily commit). Licence CC0 per Crossref's metadata licence table (checked 2026-10-26). https://gitlab.com/crossref/retraction-watch-data https://www.crossref.org/documentation/retrieve-metadata/
Rittman M (2025-01-29). Retraction Watch retractions now in the Crossref API. Crossref Blog. doi:10.13003/692016. https://www.crossref.org/blog/retraction-watch-retractions-now-in-the-crossref-api/
Shi H, Yu Y, Romero DM, Horvát EÁ (2026). The Persistence of Retracted Papers on Wikipedia. Proceedings of the ACM on Human-Computer Interaction 10(6): 1-35. doi:10.1145/3817014. Preprint arXiv:2509.18403 v3 (2026-01-26); figures checked against v3 on 2026-10-26. https://doi.org/10.1145/3817014
Nicholson JM, Uppala A, Sieber M, Grabitz P, Mordaunt M, Rife SC (2021). Measuring the quality of scientific references in Wikipedia: an analysis of more than 115M citations to over 800 000 scientific articles. FEBS Journal 288(14): 4242-4248. doi:10.1111/febs.15608. PMID 33089957. Retraction figures from the full text (PMC8060352), checked 2026-10-31. https://pubmed.ncbi.nlm.nih.gov/33089957/
Kokash N, Colavizza G (2024). Wikipedia Citations: Reproducible Citation Extraction from Multilingual Wikipedia. arXiv:2406.19291. doi:10.48550/arXiv.2406.19291. Figures checked 2026-10-31. https://arxiv.org/abs/2406.19291
RetractionBot, README ("It currently runs only on the English Wikipedia"), checked 2026-10-26. https://github.com/Samwalton9/RetractionBot
Wikidata Q14442550 (retraction template; sitelinks for de, fr, ja, zh, ko, id; none for es, ru), checked 2026-10-26. A sitelink does not show that every alias of the template is covered. https://www.wikidata.org/wiki/Q14442550

Appendix

A1. Retraction list

Retraction = a row with RetractionNature "Retraction". Papers with a "Reinstatement" row for the same DOI or PMID are dropped. Papers with only "Expression of concern" or "Correction" rows are not included.
Identifiers: OriginalPaperDOI (lowercased, trimmed; "unavailable" and blank dropped) and OriginalPaperPubMedID (0, "00000000" and blank dropped). 16 PMIDs attached to more than one paper are not used for matching.
Paper key: the DOI if present, otherwise the PMID. A paper matched by both DOI and PMID in one article is one pair.
Cochrane: DOI starting with 10.1002/14651858. or journal name containing "Cochrane Database". Each cited Cochrane paper is put in a reason group by its Retraction Watch reason codes: A (any error, misconduct, conflict-of-interest, data or publisher-investigation code), B (only "Withdrawn as Out of Date" or "Retract and Replace"), C (neither). The codes were not checked against the notices.

A2. Link extraction

externallinks rows whose domain is doi.org (including dx.doi.org): the path, URL-decoded, lowercased, with ? and # parts and trailing .,;) removed, is the DOI. PubMed rows: the first 5 to 9 digit number in the path is the PMID. DOIs on other domains (publisher sites) are not used.
Article namespace (ns=0) only, from the page XML.

A3. Detection rule (the same in every edition)

Window: the <ref> element containing the identifier (else the line) plus 300 characters each side. A pair is marked if any of its windows contains:

(i) a template call whose name contains retract (any case), Rétract, Zurückgezogen, 撤回, 撤稿 or 철회. Aliases without these strings are missed.
(ii) a word: "retract" in every edition, plus de: zurückgezogen, Rückzug, widerrufen; fr: rétract, retiré; es: retractad, retractación, retirado; ru: отозван, отзыв статьи, ретрагир; ja: 撤回, 取り下げ, 取下げ; zh: 撤稿, 撤回; ko: 철회; id: ditarik, pencabutan; sw: ubatilish (added once after the reading check found a missed Swahili word, under the plan's one-time word-list correction rule).
(iii) the DOI or PMID of the paper's retraction notice.

A4. Stage counts and intervals

Edition,Article-namespace pairs (all),Cochrane pairs removed,Main pairs,Unlocated,Located,No mark found
en,582,68,514,6,508,59
de,75,3,72,1,71,40
fr,146,16,130,2,128,78
es,144,18,126,0,126,106
ru,138,15,123,0,123,96
ja,179,19,160,0,160,103
zh,117,24,93,1,92,76
ko,49,9,40,0,40,24
id,49,16,33,0,33,20
sw,8,5,3,0,3,2

The step before this table (matched pairs on pages outside the article namespace, which were dropped) was recorded only for English: 534 matched pairs were on non-article pages. Per-edition counts for that step and the raw link totals were not kept in the outputs.

Intervals: the plan specified Wilson intervals, which assume independent pairs; pairs share articles and papers, so that assumption fails. Because the values describe all detected pairs in one fixed dump and no target population is defined, we give no intervals. Article-cluster and two-way (article and paper) bootstrap intervals were computed during the analysis and are in the summary file, but are not used in this note.

A5. Sensitivity

Cochrane items added back. 85 unique Cochrane papers are cited across the ten editions (A 21, B 58, C 6), forming 193 pairs (68 in English, 125 in the nine other editions, of which 124 are located). Share where no mark was found, with located Cochrane pairs added by reason group:

Edition,Cochrane pairs A/B/C,Main (no Cochrane),+ A,+ A and C,+ all
en,7/55/6,59/508 = 11.6%,62/515 = 12.0%,64/521 = 12.3%,92/575 = 16.0%
de,1/2/0,40/71 = 56.3%,40/72 = 55.6%,40/72 = 55.6%,42/74 = 56.8%
fr,2/14/0,78/128 = 60.9%,80/130 = 61.5%,80/130 = 61.5%,93/144 = 64.6%
es,4/13/1,106/126 = 84.1%,110/130 = 84.6%,111/131 = 84.7%,124/144 = 86.1%
ru,5/10/0,96/123 = 78.0%,97/128 = 75.8%,97/128 = 75.8%,106/137 = 77.4%
ja,5/14/0,103/160 = 64.4%,107/165 = 64.8%,107/165 = 64.8%,119/179 = 66.5%
zh,7/14/3,76/92 = 82.6%,82/99 = 82.8%,85/102 = 83.3%,98/116 = 84.5%
ko,1/7/1,24/40 = 60.0%,24/41 = 58.5%,24/42 = 57.1%,29/49 = 59.2%
id,4/11/1,20/33 = 60.6%,23/37 = 62.2%,23/38 = 60.5%,30/49 = 61.2%
8 pooled,29/85/6,543/773 = 70.2%,563/802 = 70.2%,567/808 = 70.2%,641/892 = 71.9%

(English has one unlocated Cochrane pair, Russian one; they are not in the denominators.) Pair counts by group sum over editions and count a paper once per article, so they exceed the paper counts.

Words not counted (element ii dropped): en 16.9%, de 90.1%, fr 96.1%, es 95.2%, ru 96.7%, ja 89.4%, zh 97.8%, ko 75.0%, id 78.8%. The result depends heavily on the word element.

Words near non-retracted DOIs (F4). In the same articles, every doi.org link that is not a Retraction Watch original paper or notice was searched for in the wikitext, and the rule's elements (i) and (ii) were applied to those located. Share flagged among located links: en 3.7% (356 of 9,609), de 2.3% (13/565), fr 2.6% (52/1,990), es 1.2% (23/1,904), ru 0.7% (14/2,123), ja 0.9% (29/3,368), zh 0.8% (18/2,226), ko 1.3% (17/1,260), id 0.6% (4/629), sw 0.0% (0/203; computed with the Swahili word list before "ubatilish" was added, and not rerun because the dumps were not kept). Links not found in the wikitext were excluded: en 217, ru 228, fr 41, zh 15, id 12, es 8, ko 7, ja 2, de 0, sw 0 (why they are absent was not checked; one possible reason is citations built by templates from data stored elsewhere, in which case the rule has no window to read). Counting them as unflagged gives 3.6% (en) and 0.6% (ru); counting all of them as flagged gives 5.8% (en), 10.3% (ru), 4.6% (fr) and at most 2.5% elsewhere. The plan's 5% threshold was evaluated among located links only; there, all editions are below it, so the plan's rule to drop the word element was not triggered. Under the all-flagged bound English and Russian would exceed it; the "words not counted" values above show the result for that case. F4 is the rate at which the rule fires near citations of non-retracted papers in these articles; it is not a false-positive rate for the retracted pairs, and it says nothing about missed marks.

Papers also cited on English Wikipedia (share of each edition's retracted papers that English also links): fr 40.0%, ru 44.4%, es 45.5%, zh 47.5%, ja 54.0%, de 54.7%, ko 66.7%, id 70.0%. Direction of copying is unknown.

A6. Departures from the analysis plan and post-hoc decisions

PMID "00000000" (33,806 rows) was treated as missing. Read literally it is one PMID, and the reinstatement rule then wrongly dropped 32,327 rows. No count was made with the uncorrected rule.
16 PMIDs attached to several papers were removed from PMID matching (not in the plan; effect on counts not computed).
A pair with several occurrences is marked if any window has a mark (not in the plan). This can only lower the no-mark share; the first-occurrence-only value for the main pairs was not computed (for F4 the two differ by at most 6 of 9,609 links).
F4 changed twice. (a) First pass: up to 3 non-retracted DOIs sampled per article, taken by regular expression from the wikitext, first window only. The first run counted retraction-notice DOIs as non-retracted (de 15.0%, fr 10.9%, es 5.1%, ru 7.8%); after excluding notice DOIs, en 11.8%, de 7.4%, fr 6.3% exceeded 5%, and the first pass dropped the word element for these three editions (pooled non-English share 79.3%). The sample also changed with the order in which editions were run. (b) A supplementary unsampled check took all DOIs found by regular expression in the wikitext of all fetched article pages (including pages matched only through Cochrane papers), first window only: en 3.0% (373 of 12,393), de 2.5%, fr 2.1%. (c) The final F4 (A5), as the plan specified, takes doi.org links from the externallinks table for articles with at least one main pair, excludes all Retraction Watch original-paper and notice DOIs, and uses any window. The values differ from (b) because the DOI source (externallinks doi.org links versus any DOI string in the wikitext, which includes publisher URLs), the page set and the window rule differ. Under (c) no edition exceeds 5%, and one definition is used for every edition.
Swahili: before the plan was written, a pilot had looked at Swahili (17 matched links on 9 pages, 5 of the 9 papers Cochrane items, one citation without a mark). It was included with that knowledge, marked in the plan as an instrument check rather than a comparison. After the reading check found a missed Swahili retraction word, "ubatilish" was added once, as the plan required, changing the count from 3 of 3 to 2 of 3 with no mark. After the first review it was removed from the pooled comparison.
Intervals were dropped (A4).
The Cochrane reason-group re-aggregation (A5) and a check that the matched links are bibliographic citations (93 pairs read by the AI reader, all judged to cite the paper; not a human check, and the sample record was not kept, so this is a run record that cannot be recomputed) were added after the first results.
A planned estimate of the share of citations without identifiers (hand-checking 100 random pages) was not done.
The planned check of the rule was done by an AI reader, not blind, and no human check was done.

A7. Reproducibility

Inputs: see Sources. File sizes and checksums of our downloaded copies were not recorded, except the English externallinks file (5,007,807,225 bytes, gzip integrity test passed). Wikimedia's published SHA-1 sums for the run can be compared with any fresh download while the run remains available. The Retraction Watch commit was identified after retrieval from the README's generation date (2026-10-22) and the retrieval time; the CSV's checksum was not recorded. The Retraction Watch README in the package (field definitions) is from the following day's version, generated 2026-10-23, and was retrieved later; it was not the analysed file's README.
Pages: wikitext of 1,305 matched pages was fetched by byte range from the dump server, at least 1 second apart. The Wikimedia API was not used.
Rerun from the dumps: set a working directory, then run download, extraction, page fetch, revision, the F4 script and the statistics script in the order given in the package README. Shell scripts read the working directory from an environment variable. The dumps (about 11 GB) were not kept after the analysis, so the pair-level file has not been regenerated since. The primary outcome counts, sensitivity analyses, concentration figures and final F4 figures in this note are recorded in the deposited aggregate outputs; they were computed from the pair-level file, which is not published, so they cannot be recomputed from the deposit alone. The run-record figures named above and in A4 and A6, the AI reading check and the external-source figures are not in the deposit.
The internal analysis plan is not part of the deposit; its rules are restated in A1 to A3 and A6.

Retraction marks near links to retracted papers on Wikipedia: an exploratory mechanical count in English and eight other editions