The following is a guest post from Tom Jefferson of The Cochrane Collaboration, Peter Doshi of the University of Maryland and Carl Heneghan from the University of Oxford. We asked them to tell the story behind their recent Cochrane systematic review [1] and dataset in Dryad [2] which holds valuable lessons about the evidence-base on which major public health recommendations are decided. -TJV
In the late 2000s, half the world was busy buying and stockpiling the neuraminidase inhibitors oseltamivir (Tamiflu, Roche) and zanamivir (Relenza, GSK) in fear of an influenza pandemic.
The advice to stockpile for a pandemic and also use the drugs in non-pandemic, seasonal influenza seasons came from such august bodies as the World Health Organization (WHO), the US Centers for Disease Control and Prevention (CDC) and its European counterpart, the ECDC. However, they were stockpiling on the basis of an unclear rationale, mixing the effect of the antiviral drugs on the complications of influenza (mainly pneumonia and hospitalizations) and their capacity to slow down viral spread giving time for vaccines to be crash produced and deployed.
It has since become clear that none of these parties had seen all the clinical trial evidence for these drugs. They had based their recommendations on reviews of “the literature” which sounds impressive, but in fact refers to short trial reports published in journal articles rather than the underlying detailed raw data. For example, key assumptions of antiviral performance found in the US national pandemic plan trace back to a six page long journal article written by Roche which reported on a pooled-analysis of 10 randomized trials of which only 2 have ever been published.
In contrast, each of the corresponding internal clinical study reports for these 10 trials runs thousands of pages (for background on what clinical study reports are, see here.) Despite the stockpiling, these reports have never been reviewed by CDC, ECDC, or WHO. The WHO and CDC both refused to answer our questions on the evidence base for their policies.
Our Cochrane systematic review of neuraminidase inhibitors, funded by the National Institute for Health Research in the UK, was based on analysis of the full clinical study reports for these drugs, not short journal publications. We obtained these reports from the European Medicines Agency, Roche, and GlaxoSmithKline. It took us nearly four years to obtain the full set of reports. The story of how we got hold of the complete set of clinical trials with no access restrictions is told in our essay “Multisystem failure: the story of anti-influenza drugs”.
With the publication of our review, we are making all 107 full clinical study reports publicly available. If you disagree with our findings, if you want to carry out your own analysis or if you are just curious to see what around 150,000 pages of data look like, they are one click away. Now the discussion about how well these drugs work can happen with all parties able to independently analyze all the trial evidence. This is called open science.
Be aware that there are some minimal redactions carried out by GSK and Roche. They did this to protect investigator and participant identity. While protecting participant identity is understandable, the EMA carries a different view towards protecting investigator identity: “names of experts or designated personnel with legally defined responsibilities and roles with respect to aspects of the Marketing Authorisation dossier (e.g. QP, QPPV, Clinical expert, Investigator) are included in the dossier because they have a legally defined role or responsibility and it is in the public interest to release this data”.
References
- Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya I, Mahtani KR, Nunan D, Howick J, Heneghan CJ (2014) Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database of Systematic Reviews, online in advance of print. doi:10.1002/14651858.CD008965.pub4
- Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya I, Mahtani KR, Nunan D, Howick J, Heneghan CJ (2014) Data from: Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Dryad Digital Repository. doi:10.5061/dryad.77471