Convenience sampling leads to selection bias in ED studies

8th November 2021

Convenience sampling is a potential source of sampling bias in ED studies and which is best avoided due to its impact on internal validity.

Utilising a convenience sampling (CS) approach in studies can introduce selection bias and negatively impact upon internal validity. This was the conclusion of an interesting study by a team of researchers from the Department of Emergency Medicine, St Vincent’s Hospital, Melbourne Australia. Convenience sampling represents a non-probability sampling approach in which people are included (i.e., sampled) simply because they are at the right place and time and hence a “convenient” sources of data for researchers. Emergency department (ED) studies often use CS although an important disadvantage is that it provides no guarantee that the patients recruited are truly representative of the whole cohort visiting an ED. For instance, since intoxicated patients are more likely to visit an ED during the evening or night, sampling only during the day would provide an unrepresentative sample, introduce selection bias and undermine the internal validity of the study. While a commonly used approach in research, convenience sampling has been identified as a source of bias in other areas of research such as neurology and oncology.

For the present study, researchers sought to illustrate how a CS approach could introduce selection bias on studies undertaken in an ED and hence undermine the value of such a study. They focused on four specific clinical conditions in adults: abdominal pain; chest pain; headache and dyspnoea. Using routinely available data, they retrospectively examined information from five EDs in Melbourne and extracted patient’s arrival time, which was categorised as day (8am to 4 pm), evening (4 pm to 12 pm) and night (12pm to 8 am) as well as demographics, i.e., age, gender etc. The primary outcome of interest was a patient’s discharge diagnosis at each of the three arrival times.

Findings

A total of 2,500 patients were enrolled in each of the four clinical areas. Among patients with abdominal pain there was a significant difference in discharge diagnosis across the time periods (p < 0.001) with a higher proportion of ‘unspecified/unknown’ diagnoses during the evening compared to the day (47.4% vs 41.7%). In addition, there were differences in departure status with more leaving before treatment was complete in the evening (16.7%) and night (12.2%) compared to the day (7.1%, p < 0.001).

For patients with chest pain, there was no difference in discharge diagnoses over the three time periods, although again there were differences in the departure status, with more leaving before treatment completion in the evening and night than during the day.

While there were no important differences for headache patients, for dyspnoea patients, a diagnosis of asthma was more commonly made at night compared to during the day (12.6% vs 7.5%) and more patients were admitted to the short stay unit during the day compared to the night (15.3% vs 9.8%).

Commenting on their findings, the authors stated how these results revealed significant differences in discharge diagnosis and patient characteristics across the three time periods and how using a CS to enrol patients only during office hours, would create an unrepresentative sample of ED patients. They recommended that future studies should employ consecutive sampling across all hours of the day and that CS could be mitigated by various statistical techniques such as propensity score matching. They concluded that convenience sampling should be avoided if possible or where this was unavoidable, it would be necessary to demonstrate that the use of this approach would not to have a significant impact on the primary outcome measure of a study.

Citation

Lines T et al. Nature and extent of selection bias resulting from convenience sampling in the emergency department. Emerg Med J 2021