Richard Smith, editor of the British Medical Journal (BMJ) until 2004 posted an interesting opinion piece in the BMJ last year.
I have posted it below for you to read.
Time to assume that health research is fraudulent until proven otherwise?
Health research is based on trust. Health professionals and journal editors reading the results of a clinical trial assume that the trial happened and that the results were honestly reported. But about 20% of the time, said Ben Mol, professor of obstetrics and gynaecology at Monash Health, they would be wrong. As I’ve been concerned about research fraud for 40 years, I wasn’t that surprised as many would be by this figure, but it led me to think that the time may have come to stop assuming that research actually happened and is honestly reported, and assume that the research is fraudulent until there is some evidence to support it having happened and been honestly reported. The Cochrane Collaboration, which purveys “trusted information,” has now taken a step in that direction.
As he described in a webinar last week, Ian Roberts, professor of epidemiology at the London School of Hygiene & Tropical Medicine, began to have doubts about the honest reporting of trials after a colleague asked if he knew that his systematic review showing the mannitol halved death from head injury was based on trials that had never happened. He didn’t, but he set about investigating the trials and confirmed that they hadn’t ever happened. They all had a lead author who purported to come from an institution that didn’t exist and who killed himself a few years later. The trials were all published in prestigious neurosurgery journals and had multiple co-authors. None of the co-authors had contributed patients to the trials, and some didn’t know that they were co-authors until after the trials were published. When Roberts contacted one of the journals the editor responded that “I wouldn’t trust the data.” Why, Roberts wondered, did he publish the trial? None of the trials have been retracted.
Later Roberts, who headed one of the Cochrane groups, did a systematic review of colloids versus crystalloids only to discover again that many of the trials that were included in the review could not be trusted. He is now sceptical about all systematic reviews, particularly those that are mostly reviews of multiple small trials. He compared the original idea of systematic reviews as searching for diamonds, knowledge that was available if brought together in systematic reviews; now he thinks of systematic reviewing as searching through rubbish. He proposed that small, single centre trials should be discarded, not combined in systematic reviews.
Mol, like Roberts, has conducted systematic reviews only to realise that most of the trials included either were zombie trials that were fatally flawed or were untrustworthy. What, he asked, is the scale of the problem? Although retractions are increasing, only about 0.04% of biomedical studies have been retracted, suggesting the problem is small. But the anaesthetist John Carlisle analysed 526 trials submitted to Anaesthesia and found that 73 (14%) had false data, and 43 (8%) he categorised as zombie. When he was able to examine individual patient data in 153 studies, 67 (44%) had untrustworthy data and 40 (26%) were zombie trials. Many of the trials came from the same countries (Egypt, China, India, Iran, Japan, South Korea, and Turkey), and when John Ioannidis, a professor at Stanford University, examined individual patient data from trials submitted from those countries to Anaesthesia during a year he found that many were false: 100% (7/7) in Egypt; 75% (3/ 4) in Iran; 54% (7/13) in India; 46% (22/48) in China; 40% (2/5) in Turkey; 25% (5/20) in South Korea; and 18% (2/11) in Japan. Most of the trials were zombies. Ioannidis concluded that there are hundreds of thousands of zombie trials published from those countries alone.
Others have found similar results, and Mol’s best guess is that about 20% of trials are false. Very few of these papers are retracted.
We have long known that peer review is ineffective at detecting fraud, especially if the reviewers start, as most have until now, by assuming that the research is honestly reported. I remember being part of a panel in the 1990s investigating one of Britain’s most outrageous cases of fraud, when the statistical reviewer of the study told us that he had found multiple problems with the study and only hoped that it was better done than it was reported. We asked if had ever considered that the study might be fraudulent, and he told us that he hadn’t.
We have now reached a point where those doing systematic reviews must start by assuming that a study is fraudulent until they can have some evidence to the contrary. Some supporting evidence comes from the trial having been registered and having ethics committee approval. Andrew Grey, an associate professor of medicine at the University of Auckland, and others have developed a checklist with around 40 items that can be used as a screening tool for fraud (you can view the checklist here). The REAPPRAISED checklist (Research governance, Ethics, Authorship, Plagiarism, Research conduct, Analyses and methods, Image manipulation, Statistics, Errors, Data manipulation and reporting) covers issues like “ethical oversight and funding, research productivity and investigator workload, validity of randomisation, plausibility of results and duplicate data reporting.” The checklist has been used to detect studies that have subsequently been retracted but hasn’t been through the full evaluation that you would expect for a clinical screening tool. (But I must congratulate the authors on a clever acronym: some say that dreaming up the acronym for a study is the most difficult part of the whole process.)
Roberts and others wrote about the problem of the many untrustworthy and zombie trials in The BMJ six years ago with the provocative title: “The knowledge system underpinning healthcare is not fit for purpose and must change.” They wanted the Cochrane Collaboration and anybody conducting systematic reviews to take very seriously the problem of fraud. It was perhaps coincidence, but a few weeks before the webinar the Cochrane Collaboration produced guidelines on reviewing studies where there has been a retraction, an expression of concern, or the reviewers are worried about the trustworthiness of the data.
Retractions are the easiest to deal with, but they are, as Mol said, only a tiny fraction of untrustworthy or zombie studies. An editorial in the Cochrane Library accompanying the new guidelines recognises that there is no agreement on what constitutes an untrustworthy study, screening tools are not reliable, and “Misclassification could also lead to reputational damage to authors, legal consequences, and ethical issues associated with participants having taken part in research, only for it to be discounted.” The Collaboration is being cautious but does stand to lose credibility—and income—if the world ceases to trust Cochrane Reviews because they are thought to be based on untrustworthy trials.
Research fraud is often viewed as a problem of “bad apples,” but Barbara K Redman, who spoke at the webinar insists that it is not a problem of bad apples but bad barrels if not, she said, of rotten forests or orchards. In her book Research Misconduct Policy in Biomedicine: Beyond the Bad-Apple Approach she argues that research misconduct is a systems problem—the system provides incentives to publish fraudulent research and does not have adequate regulatory processes. Researchers progress by publishing research, and because the publication system is built on trust and peer review is not designed to detect fraud it is easy to publish fraudulent research. The business model of journals and publishers depends on publishing, preferably lots of studies as cheaply as possible. They have little incentive to check for fraud and a positive disincentive to experience reputational damage—and possibly legal risk—from retracting studies. Funders, universities, and other research institutions similarly have incentives to fund and publish studies and disincentives to make a fuss about fraudulent research they may have funded or had undertaken in their institution—perhaps by one of their star researchers. Regulators often lack the legal standing and the resources to respond to what is clearly extensive fraud, recognising that proving a study to be fraudulent (as opposed to suspecting it of being fraudulent) is a skilled, complex, and time consuming process. Another problem is that research is increasingly international with participants from many institutions in many countries: who then takes on the unenviable task of investigating fraud? Science really needs global governance.
Everybody gains from the publication game, concluded Roberts, apart from the patients who suffer from being given treatments based on fraudulent data.
Stephen Lock, my predecessor as editor of The BMJ, became worried about research fraud in the 1980s, but people thought his concerns eccentric. Research authorities insisted that fraud was rare, didn’t matter because science was self-correcting, and that no patients had suffered because of scientific fraud. All those reasons for not taking research fraud seriously have proved to be false, and, 40 years on from Lock’s concerns, we are realising that the problem is huge, the system encourages fraud, and we have no adequate way to respond. It may be time to move from assuming that research has been honestly conducted and reported to assuming it to be untrustworthy until there is some evidence to the contrary.
The Naked Emperor’s Newsletter is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
The Naked Emperor’s Newsletter is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Subscriptions are best but if you would prefer to/ also like to buy me a coffee then click on the button below.
Yes and now Cochrane has fucked itself over too (h/t Alexandros Marinos).
My friend and I laugh, occasionally, about the time I was an expectant mother and read all the recommended books and even took that Lamaze course my OB/GYN insisted on.
Take it from me. In everything, you gotta figure it all out for yourself, and all too often, right in the momnet. You better be your own expert, or else.
With help from guys like you, of course, since a lot of us aren't exactly scientifically inclined. But scepticism may be the species' most important survival instinct. Cultivate it in the young toute suite.
Just read this…Cochran bought by Gates Foundation. Need to confirm.
Peter Gøtzsche for example, typically supportive of vaccination, realized how problematic the HPV vaccine was and broke with his colleagues to speak out against it. Shortly before this happened, the Gates Foundation bought out the Cochrane Collaboration (widely regarded as the most unbiased evaluators of medical evidence in the world). Gøtzsche was then expelled from the Collaboration he cofounded for speaking out against this vaccine. From substack:
thanks. I love the Midwestern Doc substack! Regarding Gotzsche, he already had quite a few problems with Cochran. His book regarding mammography is worth a read, especially if you and/or other female loved ones are still going for annual mammograms.
Be sure to pull up and save on your hard drive this 2012 Cochrane pamphlet regarding breast cancer screening, before it gets pulled off the Internet. I use this pamphlet every year to justify my refusal to succumb to mammographies:
OMG YES. It's way past time for the public to understand that ALL CONVENTIONAL MEDICINE is by definition corrupt, and certainly everything in the pharma-funded "research" (what does this word even mean in real terms?) world is worthless. WORTHLESS.
this is precisely why I enjoy your writing here so much -- you are bringing very helpful facts to light. Not pleasant but all the more reason why someone has to do it and I am grateful you are doing it.
Haven't been retracted...studies never really done...co-authors didn't know they were on the study...co-authors didn't contribute patients to the study...
I am not surprised. But the biggest frauds are the big trials overseen by Big Pharma that doctors rely upon to prescribe statins, harmful prostate procedures, destructive and damaging stents, just to name a few...
And non-trial, real world data is too often using inappropriate methodology that requires unsatisfied assumptions (see EVERY test negative study from the last year). I am becoming increasingly concerned that the entire field of medical science is snake oil
Yes, I came to conclusion of real and fraudulent intermixed within pubmed. Especially when I saw article published by three Chinese scientists credentialed in China titled mRNA safe and effective. Their review of vaers data! With all the barriers to understand ing the context of vaers data how could they be able to deduce this conclusion was my thought. Ivm and hcq use as antivirals pubs also conflicting results to help the literature appear unbiased. Psyops like these abound- like hurry up they may run out of....
We need science-literate people instead of sheeple, and open access with standardization for data collection, validation, and distribution. Standardization does not equal governance.
" Make no mistake, we are currently being assailed by a worldwide fraud of such scale and malevolence that it threatens our very existence as we know it on this planet "
Thank you so much for this, Lioness of Judah Ministry. I just printed it and will begin reading it as soon as I finish the laundry. Thank God for caring, honourable, & dedicated people in this world who work to expose evil doers crimes against humanity and to stop them from committing more atrocities against human beings and all of God's Creation. Bless you, Lioness.
This reminds me of ordering things online and having it be nothing whatsoever like the photo. Or having to cut off the land line because so many scam calls come through.
It also reminds me of a presentation at one of our departments recently. We had a parallel collaboration with Yale because they happened to be working on the same very specialized medical topic. So we're doing ours with lots of experience on this side of the ocean and they're doing the same thing over there.
Velvet curtain rolls back -- ta da! Yale had perfect, amazing, meaningful results with a specific conclusion. What a discovery! "Well, did you observe that in ours?" one of the researchers cautiously asks. "No, not at all." Universal silence and pursed lips. So out it rolls to the publishers, to redirect the course of history of medical knowledge based upon "findings" that our experts on the matter certainly never found.
These go well with the BMJ letter:
Barry Schwarz TedTalk 'Our loss of Wisdom'
Book: 'Good Work: When Excellence and Ethics Meet' by Gardner et al.
As a former Research Fellow I concur. In the research world it's not in anyone's interests to draw attention to dubious studies, so nobody does.
And yes, systematic reviews are mostly pure garbage.
Yes and now Cochrane has fucked itself over too (h/t Alexandros Marinos).
My friend and I laugh, occasionally, about the time I was an expectant mother and read all the recommended books and even took that Lamaze course my OB/GYN insisted on.
Take it from me. In everything, you gotta figure it all out for yourself, and all too often, right in the momnet. You better be your own expert, or else.
With help from guys like you, of course, since a lot of us aren't exactly scientifically inclined. But scepticism may be the species' most important survival instinct. Cultivate it in the young toute suite.
Just read this…Cochran bought by Gates Foundation. Need to confirm.
Peter Gøtzsche for example, typically supportive of vaccination, realized how problematic the HPV vaccine was and broke with his colleagues to speak out against it. Shortly before this happened, the Gates Foundation bought out the Cochrane Collaboration (widely regarded as the most unbiased evaluators of medical evidence in the world). Gøtzsche was then expelled from the Collaboration he cofounded for speaking out against this vaccine. From substack:
https://2ndsmartestguyintheworld.substack.com/p/part-ii-the-complete-history-of-depopulation?utm_source=substack&utm_medium=email
thanks. I love the Midwestern Doc substack! Regarding Gotzsche, he already had quite a few problems with Cochran. His book regarding mammography is worth a read, especially if you and/or other female loved ones are still going for annual mammograms.
Be sure to pull up and save on your hard drive this 2012 Cochrane pamphlet regarding breast cancer screening, before it gets pulled off the Internet. I use this pamphlet every year to justify my refusal to succumb to mammographies:
https://www.cochrane.dk/sites/cochrane.dk/files/uploads/images/mammography/mammography-leaflet.pdf
Oh my ! Thanks . . . I would never have known . . .
What do you mean about "h/t Alexandros Marinos?" What is the story there?
He tweeted this link.
https://blogs.bmj.com/bmjebmspotlight/2018/09/16/cochrane-a-sinking-ship/
OMG YES. It's way past time for the public to understand that ALL CONVENTIONAL MEDICINE is by definition corrupt, and certainly everything in the pharma-funded "research" (what does this word even mean in real terms?) world is worthless. WORTHLESS.
Keep emergency medicine. Burn down the rest.
“Science is self correcting”
I used to parrot this too.
Unfortunately I forgot that the corrections can take decades to centuries.
this is precisely why I enjoy your writing here so much -- you are bringing very helpful facts to light. Not pleasant but all the more reason why someone has to do it and I am grateful you are doing it.
Haven't been retracted...studies never really done...co-authors didn't know they were on the study...co-authors didn't contribute patients to the study...
I am not surprised. But the biggest frauds are the big trials overseen by Big Pharma that doctors rely upon to prescribe statins, harmful prostate procedures, destructive and damaging stents, just to name a few...
Next time I get really mad at someone I'll remember this trick of secretly co-authoring them on a fraudulent study.
And non-trial, real world data is too often using inappropriate methodology that requires unsatisfied assumptions (see EVERY test negative study from the last year). I am becoming increasingly concerned that the entire field of medical science is snake oil
"Don't listen to him he's an antivaxxer"
Yes, I came to conclusion of real and fraudulent intermixed within pubmed. Especially when I saw article published by three Chinese scientists credentialed in China titled mRNA safe and effective. Their review of vaers data! With all the barriers to understand ing the context of vaers data how could they be able to deduce this conclusion was my thought. Ivm and hcq use as antivirals pubs also conflicting results to help the literature appear unbiased. Psyops like these abound- like hurry up they may run out of....
Yes. Except for this part:
“Science really needs global governance.”
We need science-literate people instead of sheeple, and open access with standardization for data collection, validation, and distribution. Standardization does not equal governance.
I agree. We don't need more governance. Mostly we need people who care about the truth to work in science.
Each type of fraud may need to be countered in its own way. Another important one is publishing any "fail to replicate" studies.
THE COVID-19 FRAUD and WAR ON HUMANITY
" Make no mistake, we are currently being assailed by a worldwide fraud of such scale and malevolence that it threatens our very existence as we know it on this planet "
https://lionessofjudah.substack.com/p/the-covid-19-fraud-and-war-on-humanity
Thank you so much for this, Lioness of Judah Ministry. I just printed it and will begin reading it as soon as I finish the laundry. Thank God for caring, honourable, & dedicated people in this world who work to expose evil doers crimes against humanity and to stop them from committing more atrocities against human beings and all of God's Creation. Bless you, Lioness.
🙏 ❤️
This reminds me of ordering things online and having it be nothing whatsoever like the photo. Or having to cut off the land line because so many scam calls come through.
It also reminds me of a presentation at one of our departments recently. We had a parallel collaboration with Yale because they happened to be working on the same very specialized medical topic. So we're doing ours with lots of experience on this side of the ocean and they're doing the same thing over there.
Velvet curtain rolls back -- ta da! Yale had perfect, amazing, meaningful results with a specific conclusion. What a discovery! "Well, did you observe that in ours?" one of the researchers cautiously asks. "No, not at all." Universal silence and pursed lips. So out it rolls to the publishers, to redirect the course of history of medical knowledge based upon "findings" that our experts on the matter certainly never found.
These go well with the BMJ letter:
Barry Schwarz TedTalk 'Our loss of Wisdom'
Book: 'Good Work: When Excellence and Ethics Meet' by Gardner et al.
I do not understand why some of these fraudulent researchers don’t go to prison. It is rarely reported in MSM. One case in particular probably caused thousands of unnecessary deaths with 90 fraudulent papers. https://www.bmj.com/bmj/section-pdf/187846?path=/bmj/346/7900/Feature.full.pdf
Cochrane is eyewash
Yes!!!!
This is definitely something we need to be aware of. Thank you.