Substandard COVID-19 research is cause for concern

As COVID-19 claims an ever increasing number of lives, and with the onslaught of a second-wave, in the absence of a vaccine, clinicians are desperate to find any type of treatment that can help reduce symptoms and possibly save lives.

The discovery of an effective treatment will most likely bring huge financial gains for the organisation or company who succeed in finding the silver bullet. All research groups will be acutely aware of the rapid pace of change in COVID-19 research and the pressure to achieve and disseminate their findings. In their haste to get articles to press, scientists have literally inundated journals with manuscripts and ‘pre-prints’ have become much more readily available. In addition, many journals are now offering open access to their articles with the result that any initial positive results are widely disseminated in the lay media, spreading hope that these latest findings will bring salvation from the ravages of COVID-19. Moreover, a heightened desire to achieve success among the scientific community and the prospect of further grants based on positive research findings, serves only to increase the pressure upon those workers, with the attendant risk for not only an honest error but even blatant misconduct.

Nevertheless, utilising the findings of ‘pre-print’ COVID-19 research articles to aid clinical decision-making process is an inherently dangerous practice, especially as these findings have not been subjected to peer-review. In fact, the pre-print sites do caution that an article contains preliminary findings yet despite this cautionary note, clinicians sometimes appear to throw caution to the wind and seek to change clinical practice based on positive early data. This has been illustrated with the 4-aminoquinolones, hydroxychloroquine and chloroquine, mooted as treatments for COVID-19. Both drugs has been used for many years as anti-malarials and hydroxychloroquine has also proved to be of value a disease-modifying drug for the management rheumatic diseases. The original hope that hydroxychloroquine might represent a promising treatment for COVID-19, arose out of laboratory studies of its potential anti-viral activity. Animal studies showed that 4-aminoquinolones demonstrated anti-viral activity against both avian influenza A H4N1 and the Zika virus.1,2 Nonetheless, while chloroquine demonstrated in-vitro activity against influenza, it failed to provide protection against influenza in a large, randomised trial.3 At the outset of the pandemic, several small trials in China reported that hydroxychloroquine seemed beneficial and on 28 March 2020, the food and drug administration (FDA) in the US, approved the use of hydroxychloroquine for COVID-19 under its emergency use authorisation process. In May 2020, however, a large, registry-based analysis published in the Lancet that included 3016 patients receiving hydroxychloroquine, found that the drug increased the risk of death.4 Although the Lancet later retracted the paper after an independent review found that not all the necessary data was available, the FDA also revoked the emergency use authorisation for chloroquine and hydroxychloroquine on 15 June 20205. As noted by Kim et al,6 a rapid dissemination in the lay press and social media, combined with endorsement by prominent political figures, created a surge in demand for hydroxychloroquine despite an initial over-interpretation of the data and lack of direct supporting evidence of its value. This example serves to highlight our desperate need for an effective drug even where there is lack of credible evidence. A further issue which has not received much public attention is how in a number of cases, pre-print articles are never published and, in some cases even retracted.

Retracted publications
According to the US ClinicalTrials.gov site, there are currently 2548 COVID-19 clinical studies registered7 and over the last few months this has led to an enormous amount of pre-published material. Though it is inevitable that some studies fail to successful pass the peer-review process an increasing number are being retracted. According to the website ‘Retraction Watch’, currently 34 COVID-19-related publications have been retracted, three temporarily retracted, and for two articles there have been expressions of concern.8 The latter category (expression of concern) although not strictly a retraction, does alert readers that there are potential integrity issues with the paper. Although the reasons for a retraction are not always clear, it is invariably the authors themselves who instigate the retraction possibly due to more quality reporting. In fact, an analysis of COVID-19 treatment trials by researchers from the Oxford Centre for Evidence-based Medicine (CEBM) noted how too many of the current COVID-19 trials are open-labelled and too few double-blind, leading to biased results that can distort therapeutic decision-making.9

The vast number of research studies seeking approval has meant that ethics committees are deluged with COVID-19 study applications. Furthermore, these committees are unlikely to have members with the necessary specialist knowledge from key areas such as virology or immunology to critically appraise the applications. In a recently published communication in the Journal of Medical Ethics, Bramstedt10 has worryingly identified a large number of international COVID-19 studies that have been withdrawn as ‘pre-prints’ and some even after full publication.

It seems increasingly likely that the current pandemic will remain with us for several months and as new discoveries about the virus and possible treatments emerge, there will be a continued rush by researchers to publish their findings. However, it is important to avoid what has been termed ‘hot stuff bias’ in which investigators become less critical in their approach to researching a fashionable topic and journals more likely to publish findings on the topic.11

As COVID-19 continues to kill thousands of people across the globe, the research community has a duty to ensure that it undertakes high quality work that if used to inform clinical decision-making does not lead to either patient harm or provide a limited benefit. The ongoing UK RECOVERY trial demonstrates that large-scale, high quality research into COVID-19 is possible and likely to yield reliable results. Moving forward, this latter trial should serve as a template for future studies and hopefully reduce the amount of substandard research that is ultimately retracted.

References

1. Yan Y et al. Anti-malaria drug chloroquine is highly effective in treating avian influenza A H5N1 virus infection in an animal cell model. Cell Res 2013;23:300-2.
2. Shiryaev SA et al. Repurposing of the anti-malaria drug chloroquine for Zika virus treatment and prophylaxis. Sci Rep 2017;7:15771.
3. Paton NI et al. Chloroquine for influenza prevention: a randomised, double-blind, placebo-controlled trial. Lancet Infect Dis 2011;11:677-83.
4. Mehra MR et al. Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet 2020 10.1016/S0140-6736(20)31180-6.
5. FDA News Release. Coronavirus (COVID-19) Update: FDA revokes emergency use authorisation for chloroquine and hydroxychloroquine. www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-revokes-emergency-use-authorization-chloroquine-and (accessed 12 October 2020).
6. Kim AHJ et al. A rush to judgement? Rapid reporting and dissemination of results and its consequences regarding the use of hydroxychloroquine for COVID-19. Ann Intern Med 2020; doi: 10.7326/M20-1223.
7. Clinical Trials.gov. https://clinicaltrials.gov/ct2/who_table (accessed 12 October 2020).
8. Retraction Watch. https://retractionwatch.com/retracted-coronavirus-covid-19-papers/ (accessed 6 October 2020).
9. Aronson KJ et al. The ethics of COVID-19 treatment studies: too many are open, too few are double-masked.
   www.cebm.net/covid-19/the-ethics-of-covid-19-treatment-studies-too-many-are-open-too-few-are-double-masked/ (accessed 12 October 2020).
10. Bramstedt KA. The carnage of substandard research during the COVID-19 pandemic: a call for quality. J Med Ethics 2020; doi:10.1136/ medethics-2020-106494.
11. Hot stuff bias CEBM. https://catalogofbias.org/biases/hot-stuff-bias/ (accessed 6 October 2020).