The journal average that was one paper

Light grey lab-meeting slide, masthead “Quillmoor Institute · Structural methods group · Group meeting” on the left and “Where we publish · July 2011” on the right, headlined “The average recent Acta Crystallographica A paper is now cited 54 times a year. From this autumn, every paper this group writes goes there.” A standfirst says the figures are Journal Impact Factors for Acta Crystallographica Section A exactly as published in Thomson Reuters’ Journal Citation Reports; that the impact factor is the average number of citations a journal’s recent papers collect — citations received in a year to everything it published in the two years before, divided by the number of papers; and that the axis is linear from zero, every value printed, three consecutive editions. The main panel, “Acta Crystallographica Section A · Journal Impact Factor”, draws three navy bars on a zero-based axis running to 60: 2008 at 2.051, a sliver at the bottom; 2009 at 49.926, labelled “×24.3 in one edition”; and 2010 at 54.333, labelled “#2 of all journals, again”. Three cards run down the right. “The headline number” prints ×26 in large type beside “the average Acta A paper’s citations, 2008 to 2010” and says that from 2.051 to 54.333 in two editions, both years it was the second-highest impact factor in the JCR Science edition, behind only CA: A Cancer Journal for Clinicians. “Read-out for group meeting” says the group’s methods notes have been going to journals averaging two to three citations a paper, that at Acta A’s current rate the five papers planned for next year should bring in about 270 citations by 2013 before anyone has read them, and records the decision: thesis chapters, the refinement-software note and the summer students’ project all go to Acta A, with Kasia to reformat the templates. “Basis of measurement” gives the impact-factor formula and says the values are published and unrounded. A row of green ticks along the foot reads “Published JCR values”, “Zero-based linear axis”, “Three consecutive editions”, “Every value printed” and “One journal, one metric”. A footnote credits the impact factors and rankings to their sources and states that the slide is a demonstration drawn by Misleading Charts in the manner of a lab-meeting deck: the group, its papers and its decision are invented, and every figure is as published. — Showing the misleading chart

The same journal on a cream background, kicked “The same journal · every paper behind the average” and headlined “The average was one paper. Of the 6,091 citations behind the 49.9, about 5,700 went to ‘A short history of SHELX’ — the typical paper got a handful.” A standfirst explains that an impact factor is a mean, total citations divided by the number of papers; that citations are about as lopsided as data gets, so a mean of them describes no paper in particular; that Sheldrick’s 2008 review offered itself as the general citation for anyone using the SHELX programs, thousands took it up, and for the two years it sat in the window it carried the journal; and that below is every research paper Acta A published in 2007 and 2008 by citations to date. The top-left panel, “All 123 research papers, 2007–2008 · citations to date · one linear scale”, runs from 0 to 80,000 and beyond: 122 dark dots are stacked in a narrow block at zero, captioned “122 papers, every one between 0 and 343 citations. On this scale they all land within half a percent of zero, so they are stacked here in a block”; a dashed orange line just right of the block is labelled “Mean: 693”, with a note that it is drawn there so it can be seen and its true place is under 1% of the way along; and a single orange dot at the far right is labelled “A short history of SHELX, Sheldrick, 2008 · 80,652 citations · 94.6% of all 85,262 citations to these 123 papers”. The bottom-left panel, “The other 122, at a scale where they can be seen · 0 to 350 citations”, is a dot swarm piled high between 0 and 20 and thinning out to a single dot at 343, labelled “The Protein Data Bank: a historical perspective · 343”, with dashed orange lines at “Median: 12” and “Mean of these 122: 37.8”, and the note “Half of the 122 have 12 citations or fewer, eighteen years or more on. The busiest has 343: SHELX is 235 times that.” The top-right panel, “The impact factor, with the next edition”, draws four bars on a zero-based axis — 2008 at 2.051, 2009 at 49.926, 2010 at 54.333 and 2011 at 2.076 — with 2009 and 2010 shaded as the years with SHELX (2008) in the two-year window, and a small orange bar beside 2009 labelled “≈3”, captioned as 2009 minus about 5,700 SHELX citations. Two cards sit below it. “The arithmetic, as the IUCr printed it” gives 2009 as 6,091 citations ÷ 122 papers = 49.926, says about 5,700 of the 6,091 went to one paper, and that the other 121 average about 3, in line with 2008’s 2.051. “The tell” says the rise was ×24 in one edition, that in 2011, the first edition without SHELX in its window, it was back to 2.076, and that typical papers don’t multiply by 24 while one outlier joining the denominator does. Three cards run along the bottom: “What a mean of a long tail says” — that the total is large, 85,262 citations ÷ 123 papers = 693, a figure 122 of the 123 papers sit below, with 94.6% of the total in one paper; “What to quote instead” — the median, 12 with or without SHELX, the share of papers above some bar, or the whole distribution, then the outlier by name as a separate fact; and “What the group would have got” — papers sent in 2012 count in the 2013 and 2014 editions, long after SHELX left the window; almost all of the 54 was SHELX’s and none of it transferred. A footnote credits every source, says the paper-level counts are Crossref cited-by counts for volumes 63–64 regular issues with 25 non-research items excluded by us, notes that Crossref counts differ from Web of Science and that JCR counted 122 citable items, and states that both drawings are ours, the group on the first is invented, and every figure is as published or as Crossref reports it. — Showing the misleading chart

Be the first to star this exhibit

A lab-meeting slide we drew plots the published impact factor of Acta Crystallographica Section A — 2.051, then 49.926, then 54.333, outranked only by CA: A Cancer Journal for Clinicians in both years — and sends every paper the group writes there. An impact factor is a mean: in 2009, 6,091 citations divided by 122 papers. About 5,700 of those citations went to one paper, “A short history of SHELX”; the other 121 averaged about 3. In 2011, the edition after that paper left the window, the figure was 2.076. Of the 123 research papers the journal ran in 2007–2008, one now holds 94.6% of their citations and the median paper has 12.

Journal Impact Factors for Acta Crystallographica Section A — 2.051 (2008), 49.926 (2009), 54.333 (2010), 2.076 (2011) — from Journal Citation Reports as reported in Weishu Liu, Fang Liu, Chao Zuo and Junwen Zhu, “The effect of publishing a highly cited paper on journal’s impact factor: a case study of the Review of Particle Physics”, Learned Publishing (2018), arXiv:1712.03666; the 2009 calculation (6,091 citations ÷ 122 papers) from the IUCr leading article “SHELX makes an impact” (July 2010); about 5,700 and 6,700 citations to Sheldrick (2008) in 2009 and 2010, and the journal’s second place in both years, from ETH Zürich Library’s “Year of Scientometrics”. Paper-level citation counts from the Crossref REST API (ISSN 0108-7673, volumes 63–64, regular issues; 123 research items after excluding 25 book reviews, notices, reports, an obituary and an erratum), retrieved 29 October 2026, 2026Added October 29, 2026

01The claim

Acta Crystallographica Section A’s impact factor, exactly as Journal Citation Reports published it, went from 2.051 in the 2008 edition to 49.926 in 2009 and 54.333 in 2010 — up 24-fold in one edition, 26-fold in two, and outranked only by CA: A Cancer Journal for Clinicians in both years. The impact factor is the average number of citations a journal’s recent papers collect in a year, so a recent Acta A paper now averages 54 citations a year. The group’s methods notes have been going to journals that average two or three. Five papers planned for next year, at Acta A’s rate, means about 270 citations by 2013: from this autumn every paper the group writes goes to Acta A — thesis chapters, the refinement-software note and the summer students’ project included.

02The trick

Every figure on the slide is right; what goes wrong is the word “average”. An impact factor is a mean — citations in one year to everything a journal published in the two years before, divided by the number of papers — and a mean is a total shared out evenly among members who did not share it evenly. The IUCr printed the 2009 sum: 6,091 citations ÷ 122 papers = 49.926. About 5,700 of those 6,091 citations went to a single paper, George Sheldrick’s 2008 review “A short history of SHELX”, which offered itself as the general reference for anyone using the SHELX crystal-structure programs; thousands of structure papers took it up. Take that paper out and the other 121 averaged about 3 — in line with the 2.051 of the year before. No typical paper multiplied by 24. One extreme paper joined the denominator, and the mean followed it. The proof came on schedule: in the 2011 edition, whose two-year window no longer held a 2008 paper, the figure was 2.076. Citations to date show the same shape with eighteen years or more to grow. Crossref lists 123 research papers in the journal’s 2007 and 2008 volumes, with 85,262 citations between them: a mean of 693, a figure 122 of the 123 papers sit below. SHELX alone has 80,652 — 94.6% of the total. The median is 12, with or without it, and the busiest of the other 122 has 343. The group was reading a fact about one paper as a promise about all of them, and a paper sent in 2012 would have been counted in editions long after SHELX had left the window. (Both drawings are ours; the group, its papers and its decision are invented, and every figure is as published or as Crossref reports it.)

03The fix

Show the papers, not just their mean. The redraw puts all 123 research papers from 2007–2008 on one linear scale, where 122 of them crowd against zero and one sits at 80,652; then it gives the other 122 their own panel at 0 to 350, where the median of 12 and the mean of 37.8 can both be read. Beside it, the impact factor runs one edition further, to 2011, with the two editions that held SHELX shaded and 2009 recomputed without it, at about 3. The general rule: for anything with a long right tail — citations, incomes, house prices, followers — quote the median or the share above a meaningful bar next to the mean, name the outliers as separate facts, and check how far the mean moves when the biggest item is taken out. If it falls from 49.9 to 3, the mean was mostly one paper. And where a decision is about a typical member — what a new paper from this group can expect — the mean of a long tail is the wrong summary to make it on.