Randomised trial data provide definitive evidence on the extent to which a treatment or intervention works but how generalisable are the findings to the patients seen in everyday clinical practice?
The randomised, controlled, double-blind trial (RCT) is considered as the gold standard method for determining the effects of a treatment. The process of randomisation helps to eliminate any conscious or unconscious bias, whereas blinding assures that the analysis of outcomes are not prejudiced.1 An RCT is designed such that the results have external validity, in other words, that the conclusions are generalisable to clinical practice.
Since the 1990’s the terms “evidence-based medicine and evidence-based practice” have been a common theme in the medical literature and describe how clinical decision-making should be informed by current, valid and relevant evidence. Furthermore, there are numerous clinical practice guidelines that have incorporated the best available evidence to support health professionals on the assessment and management of patients.
However, the origins of evidence-based medicine appear to have arisen primarily as a means to ensure consistency of care and treatment given the wide variation observed in clinical practice.2 In addition, the rigid inclusion criteria of an RCT often means that patients who are older and those who are less able to provide informed consent because of sickness or lower levels of education are more likely to be excluded.
As a consequence, a clinical guideline even where it incorporates the best available evidence, may not reflect every day clinical practice.3 For example, guidelines focus purely on single condition whereas in reality, many patients have several co-morbidities which makes decision-making increasingly difficult, as a clinician struggles to balance the benefits and risks to the individual patient from the advice in the guideline.
One potential solution to simply relying upon the results of RCTs is to utilise the observations from real-world studies. This presents a pragmatic yet complementary approach to assessing the value of an intervention of treatment. For instance, the RCT will provide the required evidence that a treatment or intervention if efficacious, whereas a real-world study offers insight of the actual effectiveness in a clinical setting. Thus a real-world study provides information on how well an intervention performs outside of the confines of a clinical trial. In fact, the value of real world data is becoming increasingly recognised as outlined in a discussion paper by the Association of the British Pharmaceutical Industry (ABPI).4
But how does the real world effectiveness of a drug compare with the results obtained in an RCT? A recent study in JAMA Open set out to compare these differences using the latest class of oral anti-hyperglycaemic agents, the sodium-glucose co-transporter 2 inhibitors (SGLT2).5 In NICE guidance, the SGLT2 drugs are licensed for the use in type 2 diabetes as an adjunct to diet and exercise, either as monotherapy or as an add-on to other diabetic treatments.6
The JAMA study focused on empagliflozin which gained approval based on the results of four Phase III RCTs7 and was conducted in Denmark. One of the unique advantages of the country is the Danish Civil Registration system,8 in which individuals are assigned a unique identifier that allows researchers to track information on prescribed treatments and the associated outcomes.
The Danish cohort included 7034 first-time users of empagliflozin from 2014 to 2018. Interestingly, 55.1% of these individuals would have been ineligible for enrolment in the phase 3 trials which evaluated empagliflozin. The main reasons for exclusion were; noneligible glucose lowering drugs (27.8%), HbA1C levels outside of the range specified in the clinical trials (25.2%) and comorbidities (15.3%).
Nevertheless, after 6 months of treatment (the endpoint used in the RCTs), the mean reduction in HbA1C levels was -0.91% (95% confidence interval, -0.94% to -0.87%). In comparison, the pooled estimated mean reduction in HbA1C levels from the trials was -0.74% (95% confidence interval, -0.75% to -0.73%). Interestingly, the mean reduction in HbA1C levels was -1.01% (95% confidence interval, -1.07% to -0.95%) among patients who were ineligible for the trials.
These results from a real world study clearly show that the reductions in HbA1C levels were broadly comparable with those obtained from the RCTs, despite nearly half of patients not meeting the trial inclusion criteria. In addition, a group of patients in the Danish cohort were using empagliflozin as monotherapy rather than as an add-on, which was not included in the trials which indicates that the possible benefits of empagliflozin in the real world study may have been underestimated.
A further example illustrating the comparability RCT and real world data among those with type 2 diabetes was found with respect to cardiovascular outcomes. The EMPA-REG OUTCOME was an RCT designed to explore the impact of empagliflozin on cardiovascular events.9 The trial observed a significant reduction in cardiovascular events for those given empagliflozin and similar benefits were found in a real world study of those prescribed SGLT-2 inhibitors compared to other glucose lowering drugs.10
Despite RCTs having rigorous inclusion and exclusion criteria, emerging evidence from real world studies reassuringly highlights how the purported benefits observed in RCTs can still be achieved among patients with multiple co-morbidities encountered in clinical practice.
References
- Kaptchuk TJ, The double-blind, randomised, placebo-controlled trial: gold standard or golden calf? Clin Epidemiol 2001;54(6):541–9.
- Eddy DM. Evidence-based medicine: a unified approach. Health Aff 2005;24(1):9–17.
- Guthrie B et al. Adapting clinical guidelines to take account of multimorbidity. BMJ 2012;345:e6341.
- ABPI. The vision for real world data – harnessing the opportunities in the UK. Demonstrating value with real world data 2017. www.abpi.org.uk/media/1378/vision-for-real-world-data.pdf (accessed March 2020).
- Munk NE et al. Differences between randomised clinical trial participants and real-world empaglifloxin users and the changes in their glycated haemoglobin levels. JAMA Netw Open 2020;3(2):e1920949.
- NICE. Canagliflozin, Dapagliflozin and Empagliflozin as monotherapies for treating type 2 diabetes (TA390). www.nice.org.uk/guidance/ta390 (accessed March 2020).
- Dailey G. Empagliflozin for the treatment of type 2 diabetes mellitus: an overview of safety and efficacy based on phase 3 trials. J Diabetes 2015;7(4):448–61.
- Schmidt M, Pedersen L, Sorensen H. The Danish Civil Registration system as a tool in epidemiology. Eur J Epidemiol 2014;29(8):541–9.
- Zinman B et al. EMPA-REG OUTCOME investigators. Empagliflozin, cardiovascular outcomes, and mortality in type 2 diabetes. N Engl J Med 2015;373(22):2117–28.
- Kosiborod M et al. Lower risk of Heart failure and death in patients initiated on sodium-glucose cotransporter-2 inhibitors versus other glucose-lowering drugs. Circulation 2017;136(3):249–59.
