The reliability and accuracy of biomarker testing in molecular diagnostic laboratories is important for optimal patient care, because its results are used by clinicians to predict the most appropriate treatment option. To date, several biomarkers have been added to the drug labels of targeted therapies by the US Food and Drug Administration and the European Medicines Agency as a requirement prior to the administration of drugs to patients with non-small cell lung cancer (NSCLC). Technological advances led to the discovery of novel biomarkers, and to a shifting landscape from testing one gene towards testing multiple genes in parallel.1 Hence, laboratories are challenged to implement biomarkers in their routine practice in a correct and time-efficient manner.2,3
External quality assessment (EQA) schemes provide laboratories with the opportunity to verify and validate their test methods, to monitor their performance and to compare it to other laboratories worldwide.4 In addition, EQA participation is an integral part of the quality framework of diagnostic laboratories, required by the International Organization for Standardization (ISO 15189)5 and the Clinical Laboratory Improvement Amendments.6
Since 2012, the European Society of Pathology (ESP) has organised a yearly EQA scheme for biomarker analysis in NSCLC.7,8 The schemes included testing of three biomarkers, namely1 epidermal growth factor receptor (EGFR) gene variants in exons 18 to 21,2 rearrangements of the anaplastic lymphoma kinase (ALK) gene, and rearrangements in the ROS proto-oncogene 1 tyrosine-protein kinase (ROS1) gene.3
This manuscript highlights the results of the latest 2016 EQA scheme for EGFR, ALK and ROS1 analysis in NSCLC. Additionally, recommendations are made on further research on multiple health-care levels that might contribute to an improved testing quality.
Methods
The organisation of the ESP EQA schemes is performed in collaboration with the coordination centre (BQA Research Unit of KU Leuven), which is accredited according to the ISO 17043 standard for conformity assessment of proficiency testing.9 The set-up of each ESP Lung EQA scheme is determined beforehand by a steering committee of international experts in molecular diagnostics, according to the guideline on the requirements of EQA programs in molecular pathology.10
Registration was open to all laboratories worldwide. Participants received ten formalin-fixed paraffin-embedded (FFPE) samples for EGFR variant analysis and five FFPE samples for ALK and ROS1 rearrangement testing. For ALK and ROS1, laboratories could participate in a fluorescent in situ hybdrisation (FISH) and immunohistochemistry (IHC) subscheme. For FISH, five additional digital cases were provided for signal interpretation only. For IHC, participants were asked to send back their stained slides to the coordination centre, to evaluate the immunostaining quality by a team of international pathologists. The digital FISH images were available via a digital platform, along with digitised haematoxylin- and eosin-stained slides for pathologist review.
Participants were asked to analyse the samples using their routine procedures. To reflect clinical practice, the deadline for results submission was 14 calendar days after sample receipt. Scheme results were evaluated by a team of international assessors for each individual subscheme in agreement with the EQA guideline.10 Every sample was awarded two points for a correct result, and points were deducted in case of an error. Successful participation for EGFR, ALK and ROS1 was defined as a score of ≥90% of the total achievable score.
For the technical evaluation of the immunostaining, a quality score on five points was awarded and successfulness was defined as a score of ≥3/5. The average reporting score was defined by the scoring of seven different score criteria for variant analysis and IHC, and nine criteria for FISH, as defined by the guideline.10 Participants could submit data and access their results via a password-protected central database on the ESP Lung EQA scheme website (http://lung.eqascheme.org).
Results of the 2016 NSCLC EQA scheme are shown in Table 1. Results from 2012–2015 EQA schemes have previously been published.7,8,11 In total, 190 unique laboratories from 34 countries participated in 2016. Thirty-one participants from 16 countries participated in all three markers (ALK, ROS1, and EGFR).
Table 1 (click on image to enlarge)
Average analysis scores were all above 80%. For every subscheme, at least 70% of the participants were successful according to the pre-defined criteria. For the EGFR and ROS1 FISH subschemes, the percentage of successful participants was the lowest, in concordance with the higher number of laboratories analysing at least one of the samples incorrectly. On sample and laboratory levels, more technical failures were observed for ROS1 compared with other markers.
Although reporting scores were all above 80% (Table 1), only 43–52% of the participants were able to correctly include the clinical interpretation of the analysis result, depending on the marker of interest.
Discussion
Our results demonstrate decreased error and technical failure rates on sample level in 2016 compared with previous reported results.7,8 Average reporting scores were on an upwards trajectory since 2012.11 Recently, longitudinal research of EQA data has indeed confirmed that obtaining laboratory accreditation12 and frequent EQA participation8 has a positive influence on a rapid implementation of novel biomarkers in routine practice. Although evaluated for colon cancer, EQA data also shows that the methods, region and estimations for neoplastic cell estimation are highly variable within Europe.13
Despite improvement being obvious, assessing the current state of NSCLC biomarker analysis in 2016 revealed the importance of additional research to improve several indispensable elements, especially in the continuously evolving field of personalised medicine, where new biomarkers and technological advances are key and where incorrect results could lead to incorrect therapy decisions.4,10 It has been hypothesised that marker-specific challenges, such as the estimation of the neoplastic cell content for EGFR, and the recent addition of ROS1 testing to the drug label of crizotinib,8,14 might be at the base of these findings. Therefore, we recommend a multi-level research strategy with the intention to improve and maintain the quality of biomarker analysis on the long term.
Molecular diagnostic laboratories
Even though EQA schemes reflect if a sample outcome was correctly assigned, no information is available on the exact cause in case of an error or technical failure. Some errors could be due to profound problems with laboratory methodology, while others might include sample mislabelling or typing errors. Additional research is essential to evaluate the exact time point (pre-, post-, or analytical phase) of problem occurrence. Individual monitoring of EQA participants over time could reveal such critical time points in the total testing process. Moreover, this approach allows to evaluate if problems are inherent to a specific or the switch between two methodologies/sample types.
Additionally, it is useful to map the laboratories’ processes to prevent incorrect results and identify an optimal quality framework to tackle phase-, method- and sample-specific problems. A first analysis of this kind in 2016 revealed that error causes are indeed biomarker-specific, with the majority of problems being situated in the pre-analytical phase for EGFR variant analysis, analytical phase for ALK analysis and post-analytical phase for ROS1 testing (data not shown). As mentioned above, the implementation of a quality management system and regular participation to EQA schemes, have shown to improve the quality performance of your laboratory.8,11
Pathologists
The pre-analytical problems observed for EGFR analysis, can be explained by the estimation of the percentage of neoplastic cells in the FFPE samples prior to analysis, which has already shown to be highly variable between observers.15 So far, no gold standard exists. Incorrect estimations might lead to over- and underestimations, resulting in false-negative and false-positive results affecting patient safety. Therefore, large-scale data on these estimations are needed in order to define pre-set standards and guidelines to rule out a possible source of erroneous result relatively early in the test process. Additionally, reporting scores show that there is still room for improvement in correctly providing all required report elements as suggested in different guidelines and recommendations.5,10,11,16–19
The ESP and other national societies are involved in ongoing projects with experts to improve the existing documents, to educate, and to communicate the importance of correctly reporting biomarker tests to the clinicians at oncology departments.
EQA providers
The role of EQA providers is to support the laboratories in providing tools to evaluate their performance and to improve their practice. For the future it would be good to evaluate how well EQA can reflect the findings in routine molecular testing. This study can be done on two levels: first, by assessing if critical time points in the test process are identical in routine practice compared to EQA, or if specific problems could arise in routine practice.
Because EQA participants receive pre-cut slides, not all phases of the pre-analytical process (for example, cutting, mounting and labelling of sections) can be assessed at the laboratory level during EQA.2 Second, the complexity level of distributed EQA samples needs to be compared with those received in routine practice, to evaluate how specific sample characteristics might contribute to a lower quality of analysis results. In addition, schemes from different EQA providers should become harmonised to ensure equal assessment of all participating laboratories.
Clinicians
Besides the analytical results, additional research is needed to ensure quality not only within the laboratory. Clinicians exert an important role, during requesting the biomarker test1 and translation of the test results to further treatment options.2 To improve diagnostic services, it is advisable to evaluate if laboratories always receive sufficient and correct information to perform the biomarker test at the time of request, and how different laboratory structures might influence this, aiming to increase accessibility of precision medicine for clinicians and to increase patient safety.
Although reporting scores have improved and guidelines or checklists on report elements have been published, it remains unclear whether clinicians find each element useful and correctly translate the report to the treatment decision. It is logical that incorrect interpretation of an unclear report might result in the selection of an incorrect patient therapy.
Acknowledgements
We would like to thank Pfizer for the research grant and the ESP for the recognition to coordinate the EQA programs by the BQA research unit of the University of Leuven. We would like to thank the assessors, the medical and technical experts, the validating/reference laboratories, and laboratories that provided tissue material and logistic support. Our gratitude goes to all our colleagues at the BQA research unit and the ESP office for the administrative/coordination support and proofreading of this manuscript.
References
1 Hiley CT et al. Challenges in molecular testing in non-small-cell lung cancer patients with advanced disease. Lancet 2016;388(10048):1002–11.
2 Cagle PT, Allen TC, Olsen RJ. Lung cancer biomarkers: present status and future developments. Arch Pathol Lab Med 2013;2913. 137:8.
3 Cree IA et al; European Society of Pathology Task Force on Quality Assurance in Molecular Pathology; Royal College of Pathologists. Guidance for laboratories performing molecular pathology for cancer patients. J Clin Pathol 2014;67(11):923–31.
4 Tembuyser L, Dequeker E. Endorsing good quality assurance practices in molecular pathology: risks and recommendations for diagnostic laboratories and external quality assessment providers. Virchows Arch 2016;468(1):11.
5 International Organization for Standardization. ISO 15189:2012 Medical laboratories – Particular requirements for quality and competence. ISO, Geneva, 2012.
6 Clinical Laboratory Improvement Amendments of 1988, 42 U.S.C. 263a PL100-578, 1988. Laboratory Requirements, 2003, 42 C.F.R. Chapter IV, Part 493.
7 Tembuyser L et al. The relevance of external quality assessment for molecular testing for ALK positive non-small cell lung cancer: results from two pilot rounds show room for optimization. PloS one. 2014;9(11):e112159.
8 Keppens C et al. A stitch in time saves nine: external quality assessment rounds demonstrate improved quality of biomarker analysis in lung cancer. Oncotarget 2018;9(29):20524–20538.
9 International Organization for Standardization, ISO 17043:2010 Conformity assessment – General requirements for proficiency testing. ISO, Geneva, 2010.
10 van Krieken JH et al. Guideline on the requirements of external quality assessment programs in molecular pathology. Virchows Archiv 2013;462(1):27–37.
11 Tack V et al. The ins and outs of molecular pathology reporting. Virchows Arch. 2017;471(2):199–207.
12 Tack V et al. Accreditation, setting and experience as important indicators to assure quality in oncology biomarker testing laboratories. Br J Cancer (accepted).
13 Dufraing K et al. External quality assessment identifies training needs to determine the neoplastic cell content for biomarker testing. J Mol Diagn 2018 Apr; S1525–578.
14 Bubendorf L et al. Testing for ROS1 in non-small cell lung cancer: a review with recommendations. Virchows Arch 2016; 469(5):489–503.
15 Lhermitte B et al. Adequately defining tumor cell proportion in tissue samples for molecular testing improves interobserver reproducibility of its assessment. Virchows Archiv 2017;470(1):21–7.
16 Gulley ML et al. Clinical laboratory reports in molecular pathology. Arch Pathol Lab Med 2007;131(6):852–63.
17 Matthijs G et al. Guidelines for diagnostic next-generation sequencing. Eur J Hum Genet 2016; 24(1):2–5.
18 Association for Clinical Genetic Science. (ACGS) (2015) General Genetic Laboratory Reporting Recommendations. www.acgs.uk.com/media/949852/acgs_general_genetic_laboratory_reporting_r… (accessed May 2018).
19 Organisation for Economic Co-operation and Development (OECD). Quality assurance and proficiency testing for molecular genetic testing: summary report of a survey of 18 OECD Member Countries. 2005.
