MisleadingCharts
Back to the gallery

The epidemic that arrived with a new codebook

Showing the misleading chart

Be the first to star this exhibit

Between 1998 and 1999 the American death rate from Alzheimer’s disease rose 91.9%, kidney disease 33.7% and septicemia 27.0%, while influenza and pneumonia fell 31.8% — and all causes together moved 0.7%. 1999 was the first year the United States coded its death certificates to the Tenth Revision of the ICD. NCHS had coded 1996’s certificates both ways and published the conversion factor for every cause.

01The claim

American death changed shape in a single year. Every death certificate filed in the country is here, both years, age-adjusted to the same standard population by the same office — nothing sampled, nothing modelled, 2,337,256 certificates in 1998 and 2,391,399 in 1999. Total mortality barely moved: 875.8 per 100,000 to 881.9, up 0.7%. But underneath that flat total the causes rearranged themselves. The age-adjusted death rate from Alzheimer’s disease went from 8.6 to 16.5, up 91.9%, and the disease climbed from twelfth to eighth among American causes of death. Kidney disease rose 33.7%, septicemia 27.0%, hypertension 16.7%. Influenza and pneumonia lost a third of its toll, 34.6 down to 23.6. Atherosclerosis fell off the list of fifteen leading causes after at least four decades on it. Total mortality moved 0.7%; what changed is what Americans died of.

02The trick

Every figure on the slide is NCHS’s own, printed beside its bar, and the disclosure is on the slide too — in grey, under the last card: “1999 tabulations use the Tenth Revision.” 1999 was the first year the United States coded its death certificates to ICD–10 rather than ICD–9, and the two revisions do not just rename categories, they change the rules for choosing which condition on a certificate becomes the underlying cause. The largest of those, the direct-sequel rule, now treats pneumonia as a consequence of a much longer list of conditions, so a certificate listing pneumonia alongside heart disease, stroke, cancer, chronic lower respiratory disease, septicemia, malnutrition or cirrhosis is filed under the other condition instead. Alzheimer’s got a second change of its own: ICD–9 required a definitive diagnosis, and sent “Alzheimer’s-type dementia” to presenile dementia (290.1); ICD–10 relaxed that, and about 95% of Alzheimer’s gain came straight out of 290.1. None of this is guesswork — NCHS coded some 1.85 million of the 2,314,690 certificates registered in 1996 under both revisions and published a comparability ratio for each cause: 1.5536 for Alzheimer’s, 1.2320 for kidney disease, 1.1949 for septicemia, 1.1192 for hypertension, 1.0588 for stroke, 1.0478 for chronic lower respiratory disease, 0.6982 for influenza and pneumonia. The tell is the flat total the slide offers as reassurance. A recode moves deaths between headings; it cannot change how many there were. When seven headings jump by up to 92% and the total moves 0.7%, the headings are what changed. (This exhibit is our own demonstration, drawn in the house style of a mortality review from NCHS’s published tables — no such slide was published; NCHS printed the comparability-adjusted figures right alongside the raw ones.)

03The fix

Break the line where the definition broke, and where the publisher has measured the break, apply it. NCHS did the work: table 8 of the same report carries a “1998 modified” row for every leading cause, the 1998 rate recoded to ICD–10 through those ratios. Against that row the year looks entirely different. Alzheimer’s +91.9% becomes +23.1%. Kidney disease +33.7% becomes +8.3%, septicemia +27.0% becomes +6.6%, hypertension +16.7% becomes +5.0%, chronic lower respiratory disease +9.0% becomes +4.1%. Stroke’s +3.9% turns into a decline of 1.9%. Influenza and pneumonia’s collapse of 31.8% shrinks to 2.5%. All causes stays at +0.7%, because the total was never affected by any of it. Then hold the conversion factor to the same standard as the data, because it is a measurement and not a constant. NCHS says so itself about this one: the ratios were estimated on 1996 certificates, reporting of Alzheimer’s-type dementia kept climbing afterwards, and a ratio computed on 1998 data would be at least 1.69 while one computed on 1999 data “could be as high as 1.8 or 1.9” — enough to erase the remaining 23.1% as well. Its own report calls that increase “may be spurious”. So the honest chart does not claim a 23% rise in Alzheimer’s mortality either; it says this pair of years cannot settle the question, and points at the years on either side that can. The habit worth keeping is smaller than any of that: when a series steps, find out what changed that year before deciding what happened that year — and check the total, because the total is the control.