“Reckless and dishonest. Error and fraud in biological and medical research”
“Error and Fraud. The dark side of biological and medical research”
Nature of the book
I hesitate to call it a popular science book but I have tried to write a book that will sell beyond the specialist academic market. I am hoping that it would be a book that could be recommended as important background reading to students taking any course in science, psychology, medicine or even education. I am also aiming at the educated general public including teachers and health professionals i.e. the sort of people who read Ben Goldacre’s books. I want my messages to be widely read and any monetary gain would be of secondary concern to me. I have spent almost 3 years researching this book to academic level and probably have more material than needed for a single book e.g. there are probably far too many fraud case studies to include in a single book. I will shape and select from this already drafted material to fit the requirements of the publisher and target audience. I envisage a dedicated web-site where surplus material can be made available and also where new material can be added e.g. as new cases of fraud emerge. I have given a number of talks to university groups about selected aspects research fraud or error and I am keen to receive invitations for others on an “expenses only” basis.
In this introductory section I explain how my long term interest in major errors in biomedical research developed into an interest in research fraud because of personal and family experiences. I explain why I am writing the book and what I hope to achieve and give a brief introduction to the nature and characteristics of fraud and fraudsters.
Part 1 – The research process and the sources and consequences of major errors in biomedical research
There is a brief review of the various observational and experimental research methods available to researchers in the biomedical sciences and how this evidence from a wide variety of approaches should be graded when making policy or clinical decisions e.g. whether health promotion advice is justified or whether a particular treatment ifs effective. These are the sorts of judgements now routinely made by the National Institute of Care and Health Excellence (NICE) in the UK. I suggest how mistakes can and have occurred in the past, usually when scientists or doctors have acted prematurely on the basis of evidence that should properly be regarded as only preliminary.
I review four examples of what I see as important errors that have arisen because of premature conclusions being drawn from studies which are at a relatively low level in the hierarchy of evidence e.g. animal studies and epidemiological studies. I try to suggest why these mistakes occurred and the consequences of these mistakes. I have already written about these four in my past books and review articles:
- The promotion of front sleeping for babies in the 1970s and early 1980s that led to a worldwide epidemic of cot deaths which was reversed in the UK by the “back to sleep” campaign that started in 1991. This mistake resulted in many thousands of extra UK cot deaths and probably hundreds of thousands of extra cot deaths around the world.
- The almost unanimous belief in the 1950s, 60s and 70s that the world was desperately short of protein and that protein malnutrition was by far the most important cause of worldwide malnutrition. Huge amounts of time, effort and resources were expended in trying to solve this massive problem which is now generally agreed to have been an illusory one.
- The belief that antioxidant supplements given to generally well-nourished people would increase life expectancy and reduce the levels of heart disease and cancer. Clinical trials of antioxidant supplements show no evidence of benefit and some have a net harmful effect.
- The notion that human obesity might be caused by failure of a tissue know as brown fat to burn off surplus calories. This brown fat “thermogenesis” was thought to prevent or limit weight gain when people overeat. Misapplication to people of observations made in genetically obese mice made a major contribution to this mistake.
I then address the question – “Are these four case studies just isolated but important examples of scientific mistakes or are they part of a wider problem with the credibility of much of published science”? I give a concise review of a body of work claiming that:
- “Most published research findings are false” (Ioannidis, 2005 – a paper which has been viewed 1.5 million times and cited 1870 times)
- “Any claim coming from an observational study is most likely to be wrong” Young and Karr (2011)
- Most pre-clinical studies, even though published in top journals, cannot be repeated with the same conclusions by an industrial laboratory (Begley and Ellis, 2012)
- There is no attempt by other independent authors to repeat much of the research that is published and much of it is probably never read by anyone other than the authors
- It has been suggested that up to 85% of the resources devoted to research are wasted
- Much published research cannot be reproduced (e.g. the reproducibility project in psychology conducted by the Center for Open Science which attempted to reproduce 100 published psychology studies but were successful in less than a third of cases).
Scientists need to publish regularly in better journals to advance their careers and top journals favour clean, statistically significant positive data that affirms the hypothesis of the authors. This drive for publications and the need for statistically significant positive data encourages bad practice and leads to poor reproducibility of much the data that is published. I briefly explain the use of statistics and the meaning of statistical significance and evidence that scientists take biased decisions to generate diet that can be classified as statistical significant. I discuss and try to explain in lay terms many of the reasons why so much of published research has been shown to be unreliable, many of which can be considered under the general heading of bias, such as:
- Multiple analyses – testing correlations between multiple variables of differences between multiple variable until one is found that is statistically significant.
- Selective publication – only publishing results that support a particular hypothesis and ignoring data that does not.
- Selective exclusion or inclusion of outlying results which affect the statistical significance of results.
- Multiple modelling – in observational studies one corrects for variables that can affect the association being tested. Which variables are corrected for and how that correction is done can influence whether a tested association is significant or not.
- Underpowered studies – small studies tend to produce results that are widely scattered around the true results and if many different groups are working on a specific question then some may produce statistically significant, publishable results. A small studies is known to be less likely to produce a statistically significant result but any significant result produced by such a study is also less likely to be true.
- Small effect sizes – many of the headline generating claimed relationships between diet and disease have relative risks in the 1-1.5 range i.e. the high risk group is up to one and a half times more likely to develop the disease than the low risk category. This compares to 15-40 fold increased risk of lung cancer in smokers and a 3-8 times increase in cot death associated with front sleeping. In many such studies the claimed associations are probably just a good measure of the bias in the study.
Randomised controlled trials (clinical trials) and the aggregation of clinical trials into meta-analyses have been termed the gold and platinum standards of evidence in medical science. I discuss some of the limitations of these methods and discuss why they do not always produce consistent results.
Part 2 – Research fraud and the publication process
I start with full case-studies (I have 22 completed) of people who have committed research fraud and published fraudulent data in different areas of research ranging from educational psychology, to botany, to anaesthesiology, to drug research and to cancer therapy. I have not adopted a rigid format for these case studies but have tried to explain what the accused person did and to set their research into the context of scientific beliefs of their era and how they affected thinking and in some cases how they influenced policy makers and clinical decision makers. The case-study of Sir Cyril Burt is comfortably the longest because it covers many examples of deliberate fraud or unbelievable bias covering more than a century; bias that in some instances is so blatant that it was either done intentionally or in reckless disregard of accepted practices. His writings are discussed in the context of more than a century of controversial and dubious research on the role of genetics (therefore also race and sex) in the determination of intelligence. This flawed research affected educational, penal and immigration policies including the forcible sterilisation of tens of thousands of “morons” and undesirables in the USA. The South African breast cancer specialist Werner Bezwoda callously faked clinical trials which indicated that use of “high dose chemotherapy” improved survival in advanced breast cancer patient. Many tens of thousands of women underwent this debilitating, expensive and high risk therapy for no apparent benefit; Bezwoda’s falsified data was an important factor that increased and prolonged use of this treatment. His false research indirectly caused the premature deaths of many breast cancer patients. Examples of other case studies include:
- Joachim Bold – a German anaesthesiologist who faked clinical trial data which suggested that certain types of plasma expanders were beneficial in the management of critically ill patients. Use of the substances that he championed are now classified as contra-indicated in many critical care situations.
- Michael Briggs – a British born biochemist who faked data on the comparative safety of different oral contraceptive preparations. Dr James Rossiter raised concerns about the authenticity of Briggs work whilst at Deakin University in Australia. Rossiter has written about the difficulties he encountered in trying to get the Briggs’ practices investigated and a campaign of harassment and intimidation that he was subjected to after his decision to act as a whistle-blower.
- Yoshitaka Fujii – a Japanese anaesthesiologist who published extensively about treatments for post-operative nausea and vomiting. He currently holds the record for the number of retracted papers at around 190. He was able to continue publishing for more than a decade after concerns were first raised about the authenticity of his published data.
- Vishwa Jit Gupta – an Indian geologists who collected fossil specimens from shops and museums around the world and then claimed to have found them in the Himalayas even though such specimens had sometimes never been seen within thousands of kilometres of the Himalayas. He duped other scientists into supporting his claims by sending them genuine fossils and asking them to authenticate and describe them but making them believe that they had been found in the Himalayas.
- John William Heslop Harrison – a British botanist who cultivated plants, planted them on the Island of Rum in the Hebrides and then claimed to have discovered them as native plants on this small island even though they were not found anywhere else in the British Isles. He used these unique discoveries to support his claim that the Hebrides had remained largely ice-free during the last ice-age allowing these plants to survive there but nowhere else in Britain. “Painting the mice” has become a euphemism for faking research results.
- Haruko Obokata – A Japanese scientist published two papers in January 2014 in which she claimed that stem cells could be generated by simply subjecting ordinary skin cells to mild acid shock. An ethically non-controversial means for producing unlimited supplies of stem cells would have been a major medical breakthrough. These papers were retracted within a few weeks of publication. Obokata’s supervisor hanged himself in his laboratory as a result of the stress associated with the case.
- Jon Sudbo – a Norwegian dentist who falsely claimed to be able to predict which white oral lesions (leukoplakia) would develop into cancerous lesions.
- William Summerlin – an American dermatologist who claimed that he could transplant skin, corneas and other tissues without the need for anti-rejection drugs even when he transplanted between different species. He was caught darkening patches of supposedly black skin transplanted into white mice.
- Andrew Jeremy Wakefield – a British gastroenterologist who in 1998 published a paper in The Lancet claiming that there was a link between the MMR vaccination and autism. The publicity surrounding his claims led to a huge media debate about the safety of the MMR vaccination and a large reduction in the take up of the vaccine by parents.
The case studies are a personal selection, chosen because of their interest to me, because they are famous examples or because I have decided that they illustrate an important general feature. I have deliberately chosen studies from a wide range of subject areas and from around the world (although not the physical sciences). There are many others that I could have used and I have 13 further brief case summaries often illustrating just one or two key features.
The nature and extent of research fraud is discussed including formal definitions and attempts that have been made to assess its prevalence. I indicate the serious damage that some fraudsters have caused, from waste of scarce research time and funding right through to adversely affecting the treatment of sick and injured people and in some cases indirectly killing patients.
The assumption is often made that peer review is an effective protection against the publication of fraudulent data and that any rogue papers that do get published are likely to be rapidly identified when the data is scrutinized by a critical readership and/or cannot be reproduced i.e. that science is self-correcting. I discuss evidence that contradicts this rather complacent belief of many scientists. Most fraudsters have been unmasked because of the actions of a whistle-blower and many of the others because glaring flaws in their published work have been identified by critical readers. Using examples that I have researched as part of case-studies I will discuss the safeguards that should keep the literature relatively clear of fraudulent data and how effective these safeguards are in practice. What should prevent the publication of fraudulent data – protection? How is fraudulent data exposed after publication – detection? What happens to fraudulent data (and fraudulent authors) after exposure – disinfection? There is a chapter dealing with each of these three questions. Finally I suggest a series of potential measures that might reduce the level and impact of research fraud from increasing awareness in the scientific community to the possibility that willful scientific and medical research fraud might be made a specific criminal offence.
Interested in further information, please contact email@example.com
Dr Geoff is currently in search of agent/publisher for the above project.