Doctoring the evidence

‘You can cheat to get a paper. You can cheat to get a degree. 

You can cheat to get a grant.

You can’t cheat to cure a disease, biology doesn’t care.’ 

Professor Matthew Schrag, Vanderbilt University – quoted in Doctored.

Doctored: Fraud, arrogance and tragedy in the quest to cure Alzheimer’s by Charles Piller, reads in some ways like a detective novel, except this a true story. Piller, a scientific journalist gets a tip from a whistleblower about a major case of research misconduct and follows it up. This is the time when two big pharma companies are beginning to promote  the ‘disease modifying’ donanemab and lecanemab, not long after aducanumab, hailed as the ‘first drug to slow down the disease’ has been heavily promoted over the breakfast table by the Times, the BBC,  – and everyone else. I know very little about dementia research, it’s not my field, but I thought the hype was suspiciously over the top, and that proved to be right – Aducanumab was neither effective nor safe. 

But there are bigger problems. Not only have billions of dollars been spent on research, but a considerable amount of that, as Piller discovers in his meticulous investigative journey through the field, has been wasted. 

Piller is aided in his quest by Professor Matthew Schrag, the whistleblower who contacts him about suspect images and doctored research findings published in highly respected journals, and a group of other investigators, some of them expert forensic image analysts.  He uncovers a story that questions a bedrock of the research behind these new drugs – the amyloid hypothesis. The theory goes like this – Alzheimer’s is caused by an accumulation of a substance called in the brain amyloid. If this is true, then removing it would lead to a cure. That’s what the new drugs all do – except they haven’t been shown to lead to much of an improvement in the cognition of the sufferer. Somewhere, something is going wrong – maybe with the research that supported this theory in the first place and perhaps with some of the studies being carried out to develop other new drugs which don’t seem to work either.

After reviewing thousands of scientific papers and assessing a ton of research carried out over the past few decades it becomes clear that, as Piller puts it succinctly, ‘exaggeration, hype, sheer fakery and fraud,’ have characterised this field over that period. It’s shocking in the extreme, and not least because of the thousands of patients who have been entered into clinical trials based on these questionable findings. Vulnerable people who are desperate for some hope. Indeed, Piller begins his story with one such person, Stephen, who is too unwell to be entered into trials of the major new potential disease modifying drugs, because his disease is too advanced, but gets accepted for a study of another drug with much wider entry wider criteria. Too wide. It’s the concerns that have been raised about the dodgy science which get Matthew Schrag involved in the first place, and there is so much of it. Yet, as we learn at the end, so little has happened to the careers of key people pursuing grants and glory who are mixed up in this. Universities and grant giving bodies are inept at policing research fraud and editors of journals are themselves too closely enmeshed with the high-profile researchers involved to hold them to account. Research that aligns with the amyloid hypothesis has a much greater chance of getting funded. Funding brings promotion, awards, and glory. Positive results matter, and feed into the market for commercial spin-offs and making a fortune, from investors and sales. 

The writer Naomi Alderman in her brilliant book ‘don’t burn anyone at the stake today,’ says we are in the midst of a third great information crisis because of the mass of knowledge, opinions, ideas and of course misinformation brought about by the internet. This is where we are at in the field of mental health too. It’s a field that is full of noise.  Reading Alderman’s book so soon after Charles Piller’s devastating account of the impact of fraudulent science on the world of Alzheimer’s disease helped to crystallise some of my own thoughts and fears after retiring from a life in research. I’ve never been involved in drug trials. I was a primary care and mental health service researcher but even those fields are not immune. There is pressure in academia to get grants above anything else and to publish, publish, publish – work that isn’t always the highest quality. There is a pressure to get positive results that will please your funders. Reading Piller’s book me reminded of a time a few years ago when a colleague expressed his belief that another person we knew had resorted to fakery. He was very angry about it. I dismissed it. I know now that I shouldn’t have. I’ve seen researchers in all fields repeatedly report findings that fit with their assumptions, to the point that I have to question whether they are really testing a hypothesis or simply cherry-picking findings for confirmation of their prior beliefs.  Results of new interventions can be disseminated in misleading ways – often by the internet, and in the press by university publicity departments even before they have been peer reviewed. New therapies can be monetised just like medications. There is rarely a single solution to a complex problem.

When it comes to dissemination on social media Alderman discusses how easy it is to amplify dodgy findings and statements on social media such as the ill-advised conclusion Secretary Kennedy has drawn from a study of a keto diet for schizophrenia.  ‘Something I try to do today,’ writes Alderman, ‘is to notice how something I want to share on social media makes me feel. I try to use that as a cue to slow down and check my facts.’  

Maybe we should be doing the same when it comes to drawing conclusions from scientific papers and keeping in mind that the unthinkable may have to be thought. 

 My latest book: Out of Her Mind: How we are failing women’s mental health and what must change is out now.