Important discovery or bad science? Rising mortality rates in the US discussed at Journal Club

On Thursday 2nd March the NDPH Student Journal club welcomed Alastair Gray, Professor of Health Economics and Director of HERC, as our guest speaker. He presented a paper looking at the all-cause mortality of middle-aged and white non-Hispanic men and women in the US between 1999 and 2013. After aggregating national statistics, the researchers found that there was a marked increase in mid-life morbidity and mortality among US white non-Hispanics compared to US Hispanics and other countries.

All-cause mortality, ages 45–54 for US White non-Hispanics (USW), US Hispanics (USH), and six comparison countries: France (FRA), Germany (GER), the United Kingdom (UK), Canada (CAN), Australia (AUS), and Sweden (SWE).

Case and Deaton PNAS 2015;112:15078-15083

When this was broken down further by cause of death, there was a sharp increase in the number of deaths from poisonings (drug and alcohol poisonings, both intentional and accidental), chronic liver disease and suicides.

Mortality by cause, white non-Hispanics ages 45–54.

Case and Deaton PNAS 2015;112:15078-1508

Their interpretation – which was politically pertinent given that the publication coincided with the run-up to the US election – tied the results to economic insecurity, with those Americans currently in mid-life being a “lost generation” whose future is less bright than those before them, thanks to a productivity slowdown in the 1970s, slow growth in earnings and widening inequality.

Professor Gray then went on to analyse the reaction of the paper from the blogosphere and mainstream media. According to social media tracker Altmetric, in the first month it was published, the paper was picked up by 84 news outlets and 42 blogs and was tweeted 600 times. The Washington Post called the results “startling”, Public Broadcasting Service termed it “stunning” and Al Jazeera reported it as being “devastating”. (All the big words we who work in science are never allowed to use…)

It then came to the attention of economist and blogger Andrew Gelman from the Department of Statistics at Columbia University, New York. Gelman argued that Case and Deaton had not adjusted for changes in the composition of the age group 45-54. As the baby boomer generation moved through the group, the average age increased from 49.3 in 1999 to 49.7 in 2013. As you can see from the charts from Gelman below, when adjusted, this small increase in mean age does have an effect, especially when looking at the non-Hispanic white group aged 45-54 (top middle chart). The upward trend in mortality from the raw data is flattened significantly after adjustments for age have been made.

Gelman 2015

The picture also changes after splitting the population by gender – there is a marked increase in deaths for women aged 45-54, but not for men. Case and Deaton showed that when looking at cause-specific mortality, poisonings, chronic liver disease and suicide were the causes with the largest increases in rates. These are causes of death that are more strongly associated with men, rather than women, plus they only account for around one third of the increase in mortality compared to other countries – what accounts for the remaining two-thirds?

Gelman 2015

Case and Deaton acknowledged that they could have looked more closely at geographic variance and differences between gender, which were both missed out of the analysis. They also admitted that they did not fully explain that the causes of death they focussed on – poisonings, chronic liver disease and suicides – were insufficient to account for the all-cause increase in mortality that was reported in the headlines.

What does this teach us about the publication and interpretation of public health statistics, or rather, what bigger questions does this paper raise? Firstly, shouldn’t these issues have been picked up by the peer reviewers? ‘Yes’ should be the answer, although this might be a bit idealistic as it’s unlikely that all the reviewers for the Proceedings of the National Academy of Sciences of the United States of America will be statistical experts. It might be that this paper was rejected from journals with a higher impact factor before being published in PNAS.

Secondly, does the criticism this paper received make it ‘null and void’? Even Gelman admitted that the key finding that there is a contrast between mortality rates of white, middle aged non-Hispanics in the US and other countries is valid, even if the absolute increase is questionable. Despite the methods, there are clearly social, geopolitical and economic reasons behind the results that need exploring and addressing. You could even argue that the stir it caused has drawn even more attention to its results – perhaps bad news eventually becomes good news…

Following on from that, does a paper’s interpretation by the media and blogosphere matter more than the impact factor of a journal? Despite not being published in the Lancet or BMJ, the results and headlines made it into the mainstream press and caught the attention of the public, politicians and economists because its damning headlines on the state of healthcare in a supposedly developed country. Ironically impact factor isn’t the be all and end all of how much of an impact a piece of research can have, but as academics we should be striving for quality and integrity first, headlines and tweets second.

The members of the Student Research Group would like to thank Alastair Gray for his interesting and engaging presentation and discussion on rising mortality rates in the US and UK. The full details of the two journals he presented can be found below. Blog post by Lauren Bandy.