Should clinicians be sceptical about the reported findings in oncology randomised clinical trials?

8th June 2023

A recent analysis suggests changes to primary endpoints occur frequently in randomised clinical trials in oncology, representing a risk of suboptimal – or even potentially harmful – patient care, together with outcome reporting bias. Rod Tucker investigates what this means for clinicians.

Evidence-based clinical practice is a fundamental paradigm of modern medicine and of huge importance in areas such as oncology where treatments have the potential to be curative. Consequently, there is an expectation that the data derived from randomised clinical trials (RCTs) – which reflect the highest level of scientific evidence – are complete, accurate and unbiased.

Equally important in RCTs is the primary endpoint. This represents the most important outcome and serves to assess the main objective of a trial. When designing a randomised trial, a crucial principle is that the endpoints, especially the primary endpoint, is set in advance. Failure to do so, can introduce bias but, more worryingly, creates opportunities for manipulation.

A lack of transparency

The importance of setting the primary outcome prior to commencing a study and not deviating from the original protocol, was first highlighted in 1990 by Jay Siegel, a physician and research scientist working for the FDA in the US. Siegel discussed how the published description of a study could differ from the original protocol – for instance, changes to the primary endpoint, sample size or statistical tests.

Thus, in a double-blind RCT, if researchers discovered once masked allocation was revealed, that the results didn’t suit their needs, it might be possible to ‘improve’ the findings through manipulation of the data. Siegel suggested that this might happen by reporting on fewer patients or even focusing on the findings from a particular subgroup. In the absence of transparency, he felt that both the pre-publication reviewers and those reading the final published article would not be aware of any retrospective protocol modifications and thus be unable judge the reliability of the final results.

This lack of transparency in clinical trial reporting, exemplified by inadequate access to trial protocols and selective reporting in the final published results, has been suspected for many years, but there was a lack of proof that the practice was used.

This suspicion was confirmed in a 2004 analysis by researchers from Oxford. In an analysis of 102 trials, the researchers uncovered that 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. In addition, when the published articles were compared to the original protocol, 62% of trials had at least one primary outcome that was changed, introduced, or omitted.

Later work simply re-affirmed this finding. In a 2015 systematic review of registered and published outcomes in randomised trials, researchers found that discrepancies between registered and published outcomes of clinical trials were common. Furthermore, this occurred regardless of either the body funding the study or the journal in which the articles were published.

Frequency of primary endpoint changes

But did the use of selective reporting influence trial outcomes? In other words, how often did researchers alter the original primary outcomes to ‘fit’ the available data and consequently only publish positive findings?

This was the question considered in a recent study published in JAMA Network Open by a multi-disciplinary team of US researchers. Focusing on RCTs in oncology, the team examined the frequency with which changes to the primary endpoint were made (i.e. deviations from the original protocol) and, more importantly, if these changes were reported by the authors of a published study.

In addition, the team went one step further and also explored whether there was a relationship between changes to the primary endpoint and the trial outcomes. They examined possible changes using three different methods: the history of tracked changes on ClinicalTrials.gov; self-reported changes noted in the article; and changes reported within the protocol, including all available protocol documents.

With data from the inception of ClinicalTrials.gov to February 2020, researchers uncovered 755 trials, of which 19.2% (145 trials) had a primary endpoint change using at least one of the three detection methods. Somewhat concerning was how among these 145 trials, 70.3% failed to disclose the change in the primary endpoint within the published manuscript. In further analysis, the researchers found that changes to the primary outcome were significantly associated with trial positivity (odds ratio, OR = 1.86, 95% CI 1.25 – 2.82, p = 0.003). While the study did not provide ‘proof’ of selective reporting, the findings did suggest that researchers had changed the outcome of interest to, as Siegel suggested 30 years earlier, ‘improve’ the results.

There are, of course, perfectly legitimate reasons for making changes to a primary endpoint in a study. For example, a high rate of discontinuation or slow accrual of suitable patients would make it difficult to fully assess the impact of a particular intervention. It is also possible that, as the trial continues, emerging evidence from other studies may suggest a more appropriate endpoint. Nevertheless, it is incumbent on trialists to communicate such changes. Based on the current evidence, this does not appear to be a common practice.

Could RCT reporting standards be improved?

Clinical trialists have, for many years, had access to the Consolidated Standards of Reporting Trials (CONSORT) guidance, which was first introduced in 1996. This provided a checklist and flow diagram that authors should use for reporting a randomised clinical trial. Updated in 2010, the statement was further revised in 2022, providing 17 outcome-specific items that should be addressed in all published clinical trial reports, designed to increase transparency and hopefully to minimise the risk of selective non-reporting of trial results.

Although revising primary endpoints after commencement of a clinical is not unreasonable, especially in light of new and relevant scientific knowledge, the practice should remain uncommon. A further difficulty is that many high impact journals, while endorsing CONSORT, fail to ensure compliance with the guidance, as revealed in a recent analysis, with many rejecting correction letters documenting study shortcomings.

Whether reporting of changes to primary endpoints or other outcomes will be mandatory in the near future remains uncertain. As a result, with nearly a fifth of trials failing to document such changes, it seems likely that selective reporting will continue to plague trials. At the present time, it seems that full transparency is an ambition as opposed to a reality. Clinicians therefore have every right to remain somewhat sceptical about the reported findings in oncology randomised clinical trials.

Intensity-modulated radiation mono-therapy effective for nasopharyngeal cancer

26th August 2022

Intensity-modulated radiation mono-therapy is non-inferior to chemo-radiotherapy in patients with low-risk nasopharyngeal cancer

Intensity-modulated radiation mono-therapy (IMRT) for patients with stage II nasopharyngeal cancer gives the same level of 3-year estimated survival as cisplatin chemotherapy according to the findings of a randomised trial by Chinese researchers.

Nasopharyngeal cancer (NPC) is the 22nd most common cancer worldwide and globally there were more than 133,000 new cases in 2020. It is recommended that stage I disease is treated with radiotherapy alone whereas concurrent chemotherapy (i.e. chemoradiotherapy) is recommended for patients at stage II. 

Intensity-modulated radiotherapy (IMRT) is a form of treatment that enables precise conformation of the radiation dose to the target volume and may allow more normal tissue to be spared than with other techniques.

Despite the recommendation for IMRT and chemotherapy for patients with stage II NPC, there is a lack of supportive evidence for this approach. In fact, some data suggests that additional cisplatin-based chemotherapy in patients with nasopharyngeal carcinoma treated with standard radiotherapy, increases the risk of treatment-related death and severe acute toxicity, compared with radiotherapy alone.

Moreover, one retrospective, propensity matched study found that for patients with intermediate risk NPC treated with IMRT, additional concurrent chemotherapy did not provide any significant survival benefit but significantly more severe acute toxicities.

However, the authors added that prospective, randomised trials are warranted to confirm their findings.

With some uncertainty over whether addition of chemo-radiotherapy to IMRT was beneficial, for the present study, the Chinese team performed a randomised trial to provide more definitive evidence. They recruited patients with histologically confirmed stage II NPC and randomised them 1:1 to either IMRT at 100 mg/m2 every three weeks for 3 cycles or IMRT and chemo-radiotherapy.

The primary outcome of interest was 3-year failure-free survival, which they defined as the time between randomisation and disease relapse or any cause of death. There were several secondary outcomes, one of which was overall survival.

Intensity-modulated radiation therapy and NCP outcomes

A total of 341 patients with a mean age of 48 years (30% female) were randomised to IMRT alone (172) or combination therapy and followed for a median of 46 months.

The primary outcome of estimated 3-year failure-free survival occurred in 90.5% of those with IMRT alone and 91.9% of those receiving chemo-radiotherapy with a hazard ratio of 1.36 (95% CI 0.70 – 2.66, p = 0.85) which suggested that there was no difference between the two treatment options. Similarly, there was no difference in overall survival (98.2% vs 98.6%).

The researchers also collected quality of life data from approximately two-thirds of patients in each group. They found that those receiving IMRT, had a significantly better quality of life based on a lower symptom burden, as well as improved physical, social and emotional functioning. In addition, there was a significantly lower incidence of reported adverse events at grade 3 or 4 for those assigned to IMRT.

The authors concluded that among patients with low risk NPC, intensity-modulated radiation mono-therapy was non-inferior to concurrent chemo-radiotherapy.

Citation
Tang LL et al. Effect of Radiotherapy Alone vs Radiotherapy With Concurrent Chemoradiotherapy on Survival Without Disease Relapse in Patients With Low-risk Nasopharyngeal Carcinoma: A Randomized Clinical Trial JAMA 2022