Why research fraud really does matter

Some basic definitions

According to the US National Science Foundation (NSF):

Research misconduct is defined as fabrication, falsification or plagiarism in proposing, performing, or reviewing research or in reporting research results.

This must be done intentionally or in reckless disregard of accepted practices. They go on to define fabrication as being something that involves total invention of a data set and falsification as manipulation of data so that the research is not accurately represented in the research report. Plagiarism is defined as:

“The appropriation of another’s ideas, processes, results, or words without giving appropriate credit”

Are scientists too complacent about research fraud?

The quote below is from Dr David Taylor taken from written evidence he gave to a House of Commons committee investigating scientific peer review and cited by Brian Deer in The Guardian newspaper.

It is important to recognise that in the long term it matters little if published material is inaccurate, incompetent or even fraudulent since the advance of the scientific canon only uses that material which turns out to fit the gradually emerging jigsaw. Incorrect pieces may find a place for a time, but will always be eventually replaced by the correct piece

Whilst this may be strictly true if one takes a long enough view and ignores short or medium term collateral damage, the comments are inconsistent with some of the consequences of research fraud and error discussed in the case studies on this blog. The protein gap case study and the cot death/sleeping position case-study provide clear evidence of major harm done by scientific mistakes. Do the excess deaths of tens of thousands of babies or the waste of billions of pounds of research and international aid resources matter little?

Some of the harmful consequences of research fraud are listed below, research fraud:

  • Undermines the credibility of science and public trust in scientists
  • Is a gross misuse of fraudsters paid time and the resources of the host institution and the research funder
  • Can damage the careers and lives of collaborators and research students
  • Can slow research progress if effort and resources are channelled into unproductive areas
  • Wastes the time, effort and resources of others working in the field who may try to replicate the falsified research data or build upon it
  • Can distort the outcome of meta-analyses ; the weighted amalgamation of similar studies to give a consensus of their findings
  • Causes health policy, choice of medical treatment or public behavior to be based upon false data and false conclusions
  • Can harm patients and in some cases can be responsible for unnecessary patient deaths.

Each of these points is illustrated several times over in the fraud case studies on this blog and some of these examples are highlighted in the following paragraphs.

Undermining of credibility and trust

Every time the popular media publicises a case of research fraud, it undermines a little further the credibility of scientific research and research scientists in the eyes of the general public. As a practising scientist, even I now consider the possibility of data fabrication or manipulation whenever I read media reports from press releases that suggest unexpected or unusual research findings. In the past I would have only considered the possibility of inadvertent flaws in the design or execution of the study or maybe exaggeration/unjustified extrapolation in the interpretation of the study results. When scientists who have become celebrities because of their work or those who have received major honours are publicly disgraced then this must undermine public and political confidence in the integrity of the research community and the opinions of scientists.

The Dutch social psychologist Diederick Stapel has been called a “rock star” scientist because of his many media appearances and because of the “sexy” themes and findings of some of his fabricated research which were frequently reported in the mass media. His unmasking as a blatant fraud also attracted great publicity particularly in Holland and led some people to question the value and credibility of social psychology research generally.

Ranjit Chandra, the Canadian physician and nutrition researcher was awarded the Order of Canada in 1989, allegedly twice nominated for a Nobel Prize and was on the Maclean’s magazine honour roll of Canadians who had “made a difference” for 1995 (the year after an internal inquiry at Memorial University found that he had probably fabricated clinical trial data). In 2006 he was the subject of three broadcasts made by the Canadian Broadcasting Corporation “The secret life of Dr Chandra” in which he was openly accused of multiple acts of research fraud over many years. What if Dr Chandra had actually been awarded the Nobel Prize, the ultimate world accolade for research scientists, after it was known by his employer and some colleagues that he had committed research fraud?

In the more distant past, the English botanist John W Heslop Harrison became a Fellow of the Royal Society in London (FRS) and the educational psychologist Sir Cyril Burt was knighted in both cases because of a body of work that included much that was fabricated or falsified.

At the end of January 2014 two papers in Nature, the world’s premier science journal, made media headlines with claims that simply exposing ordinary skin cells to mild acid shock would convert them into stem cells. This “discovery” by a young Japanese postdoctoral researcher, Haruko Obokata would have had major practical and theoretical implications as this comment on the BBC web-site makes clear:

“A major scientific discovery……..It will make a fundamental change in how scientists perceive the interplay of environment and genome”. Dr Dusko Ilic (King’s College London)

Unlimited supplies of perfectly tissue-type matched stem cells could be produced from a small sample of a patient’s skin. Within a few weeks questions were raised about the authenticity of the data and discussed in the science media and in July 2014 the BBC and other news outlets reported that this report of a “major breakthrough” had been retracted because of falsification or fabrication of data by Obokata.

Wasted research money and resources

Large amounts of money from research institutions, charities, government agencies and private companies have been wasted supporting the supposed work of authors who were actually fabricating data rather than generating real data with the funds provided. In some instances fraudulent researchers have misappropriated funding to line their own pockets.

Scott Reuben the American pain specialist was ordered by the US courts to make restitution payments of $400,000 including $360,000 to drug companies; money that had been awarded to him to conduct research into their drugs which had never actually been conducted. In January 2012, The University of Connecticut returned government grant money of $890,000 that had been awarded by the US National Heart and Lung institute to Dipak Das for work relating to the heart benefits of resveratrol found in red wine.  In 2005, world renowned obesity researcher Eric Poehlman of the University of Vermont acknowledged that he had used fabricated data in 17 funding applications to the National Institute of Health (NIH) in the USA and in a later plea bargain pleaded guilty in a court in Vermont to defrauding the NIH of $547,000. Prosecutors believe that he actually defrauded the NIH of $2.9million.

The fraudulent papers of Norwegian dentist Jon Sudbo in which he claimed to be able to predict which benign oral lesions would develop into oral cancer were instrumental in persuading the US National Cancer Institute to fund an oral cancer prevention trial in Scandinavia at a cost of $300,000 per year; luckily Sudbo was unmasked before this study was fully underway.

Compare these large sums to the £50 (£1500 today) awarded by the Council of Trinity College Cambridge  to classical scholar and accomplished botanist John Raven in 1948 to travel to the Hebridean island of Rum to investigate and report upon the suspected fraudulent research activity of botanist John W Heslop Harrison.

Adverse effects upon collaborators

Probably the most extreme example of a collaborator’s life being affected by association with someone accused of serious research misconduct is that of the respected Japanese stem-cell researcher Yoshiki Sasai. He hanged himself in his laboratory in August 2014 just a few months after the retraction from Nature of the two stem cell papers that he had co-authored with Haruko Obokata. In suicide notes left by Sasai he said that the media pressure following this scandal had driven him to take his own life. An internal investigation by a RIKEN institute committee found that Obokata was solely responsible for all acts of research misconduct and exonerated Sasai in this respect but it did find that he had grave responsibility for failure of oversight. Dr Sasai was heavily criticised in the Japanese press and there were intrusive articles about his private life and his motivation for publicising Obokata’s false findings. He was also heavily criticised by a RIKEN reform committee for cutting corners in the recruitment of Obokata and in the way in which he exploited the publicity surrounding publication of her findings.

The investigating committees in Holland found strong evidence that ten of the doctoral theses produced by students working with Diederick Stapel contained chapters with fraudulent data generated by Stapel. The students were allowed to keep their PhDs but with the stipulation that it should be made known publicly that these chapters contained fabricated or highly suspect data. One unfortunate student who had just submitted her thesis felt obliged to withdraw it before her thesis defence and she still did not have a PhD three years later.

There has been a good deal of speculation suggesting that the immunologist Robert Good might have been awarded a Nobel Prize were it not for his association and co-authoring of papers with William Summerlin. Summerlin falsely claimed to have a way of being able to transplant tissue between unrelated individuals or even between different species without the need for immunosuppressive drugs. The first sentence of Good’s Wikipedia entry confirms his reputation:

Robert Good “was an American physician who performed the first successful human bone marrow transplant between persons who were not identical twins and is regarded as a founder of modern immunology.”

Professor Geoffrey Chamberlain’s acceptance of gift authorship on a fabricated report written by disgraced gynaecologist Malcolm Pearce effectively marked a rather ignominious end to his previously illustrious medical career. Chamberlain was at that time Pearce’s head of department at St George’s Hospital, president of the Royal College of Obstetricians and editor of the British Journal of Obstetrics and Gynaecology. When criticised for his acceptance of gift authorship of this paper Chamberlain felt obliged to resign from all of his professional positions.

It has been suggested that the eminent geologist Sir Arthur Smith Woodward FRS spent the last 21 years of his life trying unsuccessfully to unearth further fossils and artefacts from the area where Charles Dawson had supposedly found the Piltdown man skull.

Slowing research progress and wasting the time and resources of others

It is very difficult to even attempt to quantify how much time and effort has been wasted due to the fraudulent data published by research fraudsters. Even before the work was found to be fraudulent, an editorial in the American Journal of Physiology bemoaned the amount of time, money and energy that had been expended by other groups around the world in trying to replicate the falsified data of Jatinder Ahluwalia. This data suggested that an influx of potassium ions through a specific membrane channel was a key factor in enabling neutrophils to kill ingested pathogenic organisms; a channel that was not even present on these cells. It has also become clear that a considerable amount of time and effort was expended within the host establishment (UCL) in trying to confirm the reported findings of Ahluwalia when other groups made known their failure to replicate the findings. This data was also used to throw into question the likely benefits of antioxidants from food, supplements or even drugs in preventing chronic disease.

Although not a case of research fraud, the yogurt and ovarian cancer case study provides a striking example of how a shred of flawed and weak evidence to support a flimsy hypothesis can lead to decades of fruitless research effort. Very weak and flawed evidence of a possible link between eating yogurt and ovarian cancer published in 1989 spawned dozens of follow-up studies over four decades without any furthering of our understanding.

Treatments, policies or public behavior based upon fraudulent evidence

If one is seeking examples of policy and practice being influenced by fraudulent research then the Cyril Burt and the Heritability of Intelligence case-study is a rich source. In this case-study there is strong evidence that deeply flawed or fabricated data influenced American views and policies on:

  • Immigration into the USA
  • The treatment of offenders
  • The care and education of those classified as educationally subnormal or morons as they were termed
  • The education and treatment of black and other minority ethnic groups.

A BBC web-site news report (27/02/2015) indicates that people are still suffering from the aftermath of such policies. The US state of Virginia had just agreed to pay compensation to the few remaining victims of the state’s forced sterilisation programme who were sterilised because they were deemed to be undesirable or mentally unsound. Virginia’s Eugenical Sterilisation Act remained in force from the 1920s through to 1979 and was said to be the model for Nazi Germany’s eugenics policies. More than 8000 Virginians were forcibly sterilised during this period. 33 US states had similar forced sterilisation statutes and, according to the BBC 65,000 Americans, were forcibly sterilised under such programmes. The notion that intelligence and criminal tendencies were largely inherited was supported by a mass of biased and fraudulent data and this provided the scientific justification for such eugenics programmes which aimed to prevent morons and the criminally inclined from polluting the nation’s gene pool.

Was the belief that intelligence is largely an inherited phenomenon partly responsible for the educational segregation of children at 11 years by the 11+ examination in the UK? Because as Cyril Burt eloquently put it:

“A definite limit to what children can achieve is inexorably set by their innate capacities”.

So it seemed logical to focus academic educational effort upon those children who were the most genetically well-equipped to benefit from it (i.e. who pass the 11+ exam) and channel those who were less genetically equipped for academic studies into more practical pursuits.

Perhaps the most disturbing recent example of fabricated data encouraging false policy and clinical decision-making and also hindering progress in medical research is work of  Werner Bezwoda. His publications suggested that high dose chemotherapy (HDC) extended life expectancy in cases of metastatic breast cancer as compared to conventional dose chemotherapy and even produced an effective cure in about 30% of his patients. This data was an important reason why it became difficult to recruit subjects into clinical trials comparing HDC and conventional chemotherapy because patients and their advisers did not want to risk them being allocated to the conventional therapy control group. This delayed the completion of these clinical trials by several years and it delayed the time it took for it to become clear that this HDC offered no advantages over conventional therapy. Some US states made it mandatory for health insurers to cover the cost of HDC treatment despite no sound evidence that it was more effective than much cheaper and safer conventional chemotherapy. At the (1999) meeting in Atlanta, Georgia where the disparity between Bezwoda’s clinical trial results and those of other groups in Europe and the USA became strikingly clear, it was even suggested that new clinical trials should be conducted to test whether these differences occurred because Bezwoda had used HDC as a first line of treatment whereas the other groups had used it as a last resort treatment. This could have resulted in a whole new series of expensive, gruelling and, for some participants, life-ending clinical trials.

For almost three decades, the Indian geologist/palaeontologist Viswa Jit Gupta produced hundreds of false reports of fossils he reported having found in the Himalayas but which he had actually acquired from shops or museums around the world. These fossils had never before or since been found anywhere near the Himalayas. He often duped other experts into authenticating and describing these fossils and co-authoring his papers. These respected co-authors thus unwittingly added credibility to his falsifications. It has been suggested that all of the papers co-authored by him about such findings should be disregarded and extreme caution exercised when using syntheses and reviews that have used his data. This caused major confusion in establishing a paleontological database for the Himalayas and for those trying to use this database to make judgements about the history of the flora and fauna and maybe even geology of this region.

In the early editions of several of my books, my conclusions about the likely benefits of vitamin and mineral supplements on the immune function and infection risk of elderly people were influenced in their favor by the now disregarded and probably fabricated data of Ranjit Chandra.

Scott Reuben’s fabricated data on the benefits of the COX-2 type of painkillers increased sales of these compounds by several billion dollars worldwide and influenced the pain management protocols of many physicians and surgeons around the world. Safety concerns have led to withdrawal or severely curtailed use of some of these drugs.

The fabricated publications of psychologist Stephen Breuning on the most effective treatment of retarded children with hyperactivity had a major influence upon the way children with this problem were treated. Use of stimulant drugs like Ritalin for managing such children was greatly increased by the favorable impression given by his false data. His fabricated findings were a major influence upon the official protocols used for treating these children.

The fabricated data of biochemist Michael Briggs on the relative safety of different types of oral contraceptives gave a major and unjustified stimulus to the widespread use of so-called tri-phasic pill regimens and particularly to the use of the compound levonorgestrel which was present in early tri-phasic preparations.

Jon Sudbo’s falsified data suggested that it was possible to accurately predict the risk of benign oral lesions (leukoplakia) developing into oral cancer. This made it seem practically feasible and ethical to test the possible benefits of anti-inflammatory COX-2 inhibitors for preventing oral cancer in apparently very high risk patients. As noted above Sudbo’s fraudulent behaviour was unmasked before an expensive planned clinical trial was fully underway.

Harm to patients

Andrew Wakefield’s false data suggesting a link between the MMR vaccine and autism led to a loss of public confidence in the vaccine and a decline in vaccination rates and loss of “herd immunity” in Britain and in other countries. Despite being struck off the UK medical register he has become a folk hero of the anti-vaccine movement. Multiple outbreak of measles in Europe and the USA have resulted from reduced vaccination rates and this has been partly attributed to the effects of his discredited false data; the effects of poverty or conflict have also had a major negative impact on vaccination rates and led to major measles in countries like Ukraine. Measles is an unpleasant disease which has long term consequences for some patients and has significant mortality.

Some of the examples discussed earlier in this piece, involve patients being adversely affected by research fraud because it led to them not being given the most effective or appropriate treatment. Some extra patient deaths would almost certainly have resulted from this failure to prescribe the most effective treatment in some of these cases. The fraudulent publications of Joachim Boldt and Werner Bezwoda almost certainly contributed to extra patient deaths because they encouraged use of inappropriate treatments in very seriously ill patients.

The fabricated data of German anesthesiologist Boldt helped to increase and prolong the use of certain hydroxyethyl starch preparations (HES) as fluid replacers in seriously sick or injured people and as pump priming solutions for filling heart-lung machines during cardio-pulmonary bypass procedures. Such preparations are now listed as contra-indicated for both of these situations by regulators in both the USA and Europe. They are associated with increased risk of adverse events and with higher risk of death. The highly positive falsified data of Boldt probably contributed to extra patient deaths by encouraging and prolonging their unjustified use (see further comments relating to this case in the next section).

The false clinical trial data of Werner Bezwoda on the benefits of HDC for metastatic breast cancer increased the number of women around the world who were treated in this way. Bezwoda’s 1995 trial was the only highly positive, controlled trial of this procedure. There is significant treatment mortality associated with HDC and extra side-effects when such high doses of the cytotoxic drugs were used. Many women suffered extra side effects and hundreds or perhaps even thousands died as an indirect consequence of Bezwoda’s falsified data. More directly, a small phase II trial that sought to replicate Bezwoda’s published treatment protocol was started in Seattle, Washington. Four of the six patients recruited before the trial was abandoned, suffered serious and permanent cardiac damage (heart failure) because of the treatment and two died as a result of this.

The impact of fabricated data on meta-analyses

Meta-analysis, the technique of combining together similar studies to effectively make a single larger study of greater statistical power and authority is vulnerable to distortion by the inclusion of fabricated data. If results are being fabricated then such studies are easy to produce and also likely to appear larger and to give more clear-cut results than studies where real data has been collected; this means that they can end up being heavily weighted and seriously distorting in any meta-analysis. Meta-analysis has become an immensely popular technique with, at the latest count, 26 new meta-analyses published every day. It is also highly rated by regulators of new treatments; meta-analyses of randomized controlled trials is at the top of pyramids or numerical grading systems for evidence. This means that if fraudulent studies distort meta-analyses they are distorting what is regarded as the pinnacle of evidence in clinical and policy decision-making.

In 2005 Alia El-Kadiki and Alexander Sutton performed a meta-analysis to assess the effectiveness of micronutrient supplements in preventing infections in elderly people which was published in the BMJ. They were clearly unaware of the controversy surrounding Dr Chandra’s work and given their expertise as a surgical registrar and a statistician this is not surprising. They found eight trials that met their inclusion criteria of which two were written by Chandra and one was in the name of AL Jain which is assumed to be a Chandra pseudonym. These three papers have been the subject of much criticism and are now disregarded by other experts. El-Kadiki and Sutton analysed their data using three major measures of outcome because different trials had used different outcome measures: mean number of days of infection over a year, likelihood of at least one infection during the study period and incidence rate of infections. Only the three dubious Chandra studies had used the first outcome measure and the Forest plot of these data (Figure 1) indicates that all three studies showed a significant positive effect of the supplement and overall the result indicated a large and highly significant preventive effect of the supplement. Even though unaware of the allegations relating to these papers, El-Kadiki and Sutton clearly had some concerns about these data, noting in particular the extremely low standard deviations for this outcome. They suggested that the standard deviations might in fact be standard errors and re-analysed the data making that assumption but still found a highly significant beneficial effect of the supplements.

Figure 1           A Forest plot of trials of the effects of micronutrient supplements on the outcome measure “mean number of days of infection”. After El-Kadiki and Sutton (2005) BMJ 330, 871-877

el-kadiki-1st

In their meta-analysis for the second outcome measure, none of the three studies used by El-Kadiki and Sutton were from the Chandra group. The Forest plot for these data (figure 2) gave no indication of any positive effect of the supplements; not even a non-significant trend in that direction. It is noticeable how much wider the 95% confidence intervals are for these studies than those attributed to Chandra in figure 1. The Forest plot for the third outcome measure (not shown) indicated a pooled effect that although just on the beneficial side did not come close to reaching statistical significance. This latter plot used a total of four studies including just one of the dubious “Chandra group” studies which was the only one to indicate a significant beneficial effect.

Figure 2           A Forest plot of trials of the effects of micronutrient supplements on the outcome measure “likelihood of an infection during the study period”.  After El-Kadiki and Sutton (2005) BMJ 330, 871-877

el-kadiki-2nd

El-Kadiki and Sutton were made aware of the doubts about the validity of the three studies by Chandra and Jain soon after their meta-analysis was published and rapidly issued a correction in which they showed the effect of excluding this suspect data and essentially concluded that there was no indication of any beneficial effect of supplements on holistic outcome measures. Audrey Stephen and Alison Avenell published a similar meta-analysis in 2006 and they were clearly aware of the concerns about the papers of Chandra and Jain. They found no significant effect of supplements on their outcome measures when the results from the Chandra/Jain group were excluded. When looking through the tables in this paper it is striking how the inclusion of papers from Chandra’s group affects the probability values. Four values of p<0.00001 were obtained when one or more of these was included but no other value more significant than p<0.02 for any outcome and most were not significant.

In the Fujii case-study, accusations of data fabrication were implied by Peter Kranke and his colleagues in 2001. They published a meta-analysis in which they showed that the apparent effectiveness of the drug granisetron in preventing post-operative nausea and vomiting (PONV) was greatly distorted by what they termed a dominant centre i.e. all papers in which Yoshitaka Fujii was a co-author. This analysis was dismissed at the time and it took another 11 years before Fujii was finally unmasked as a prolific fabricator of clinical trial data. A further meta-analysis conducted after it was confirmed that his data was largely fraudulent, showed that the effectiveness of different regimens for the treatment of PONV was greatly influenced by whether or not Fujii’s fraudulent data was included.  In this case, meta-analysis was used to try to detect fraud by Kranke et al by illustrating how it changed the conclusions of a meta-analysis.

Finally, in the Joachim Boldt case-study Ryan Zarychanski and colleagues reported in 2013 in the JAMA that the hydroxyethyl starch (HES) preparations favored by Joachim Boldt’s fabricated data were not associated with a decrease in mortality compared with other resuscitation solutions. Their most disturbing observation was that when they excluded seven studies by Boldt, the use of HES was associated with significant increased risk of mortality and acute kidney injury. This means that excluding Boldt’s results moved the conclusions about HES from no benefit to net harm. This means that when his work was excluded, clinicians who already used HES would have seen a strong case for ceasing to use it and any considering switching to use of HES should have been dissuaded from doing so.

Referencing

References to primary sources are mostly not given here but they may be found in the other blog articles to which hyperlinks are included in the text.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s