Article Text

Download PDFPDF

Potential for bias in waiting time studies: events between enrolment and admission
  1. B Soboleva,
  2. P Brownb,
  3. D Zeltb
  1. aQueen's Health Policy Research Unit, Queen's University at Kingston, Canada, bDepartment of Surgery, Queen's University at Kingston
  1. Dr Sobolev, Queen's University, 3rd Floor Abramsky Hall, Kingston Ontario K7L 3N6 Canada (bs9{at}


STUDY OBJECTIVE To demonstrate the effect of exclusion of data on delays in scheduling operations in calculating difference in admission rates between two enrolment periods.

DESIGN A prospective cohort study; outcome measure—waiting time for elective admission; study variables—enrolment periods, before 31 March 1997 and after that date; the time of scheduling delay; gender; age; urgency, and type of surgery.

SETTING An acute care hospital in Ontario, Canada.

PARTICIPANTS 1173 consecutive cases accepted for elective vascular surgery between 1 July 1994 and 31 March 1999.

MAIN RESULTS Before adjustment for scheduling delays, a 20% lower admission rate was associated with period 2, rate ratio (RR) = 0.8 (95% confidence intervals (CI)= 0.7, 0.9). The difference between the periods became only marginally significant after the adjustment, RR = 0.9 (95% CI=0.8, 1.0). No difference between the periods was found when admission rates were compared before a delay occurred, RR = 0.9 (95% CI=0.8, 1.1). In delayed patients, those enrolled in period 1 and 2 had, respectively, a 40% and a 60% lower admission rate than the period 1 patients admitted without scheduling delays, RR = 0.6 (95% CI=0.4, 0.8) for period 1 and RR = 0.4 (95%CI=0.3, 0.5) for period 2.

CONCLUSIONS The results provide evidence that patients experiencing a delay in scheduling operation have a lower admission rate after the event. Thus, potential for bias exists when between group comparison of waiting time is done without adjustment for an intermediate event that may occur before elective admission.

  • elective admission
  • between group comparisons

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Time to elective admission is often compared between providers, periods, fundholding plans, or management practices, to evaluate policy and performance. In these between group comparisons, there is a potential for bias if patients may experience an intermediate event occurring between enrolment and admission for treatment. These include a delay in scheduling the operation, cancellation of booked services, or reinstatement after self deferral. The occurrence of the events preceding admission may alter the admission rate in one group, having no effect in the other; have the same effect in both groups; or have different effects in each group. The common concern is, therefore, a bias in estimates of relative admission rates when comparison is done without considering events preceding admission. Some argue that, as in the case of factors fixed at the start of a prospective study, adjustment should be done for time varying covariates when studying difference between groups.1 The primary purpose of this adjustment is to see how the effects of fixed factors change when intermediate events occur.2 In addition, including the timing of the relevant events may help strengthen the inferences on how the intermediate event affects the admission rate.3

To assess the magnitude of the bias when access to surgery is compared between two enrolment periods without data on intermediate events, we used data of waiting times for elective vascular surgery collected prospectively at an acute care hospital in Ontario, Canada. The aim was to demonstrate the effect of exclusion of data on delays in scheduling operations in calculating difference in admission rates. Primary comparisons were between patients accepted for treatment during two enrolment periods in which availability of some hospital resources was not uniform. Waiting time analysis was conducted in two ways: firstly, with data on the main event, elective admission, and, then, with additional data on the occurrence and time of an intermediate event, delay in scheduling operation. We examined four elective surgical procedures for treatment of diseases of the arteries and veins: abdominal aortic aneurysm repair, carotid endarterectomy, peripheral vascular bypass surgery and other procedures involving blood supply to the legs, and vascular access for long term renal failure.



Data were taken from a waiting time registry set up to prospectively collect data on waiting times for elective admission in a cohort of patients accepted for vascular surgery in the Department of Surgery, Queen's University, Kingston, Canada. The registry records include all patients referred to the department for outpatient clinic assessment and then added to the waiting list. Not included were patients seen on an emergency basis outside the clinic, and patients who underwent emergency surgery.


Patients are added to the waiting list after a consultation visit in which surgery is deemed necessary. The date of the surgeon's letter to the patient's referring primary care physician regarding the acceptance for surgery serves as the date of enrolment on the list. Patients are removed from the list if they reconsider decision for surgery, if they die while awaiting surgery, if their conditions deteriorate so that the operation is no longer possible, if their conditions improve and make the surgery unnecessary, or when surgery is done.

Scheduling patients for surgery may be delayed if their conditions prevent the operation; if they decide to postpone surgery; if hospital ward or intensive care unit bed, or operating room time, is unavailable upon scheduling surgery; if their doctors decide to send the patient for additional preoperative investigation.


Each patient has a waiting time calculated in weeks from enrolment to admission or to removal without surgery. On the basis of internal guidelines for the number of days a patient with certain type and severity of condition can safely wait for surgery (maximum recommended delay), each patient was assigned an urgency status to determine their relative position on the waiting list.4


Patients were divided in two groups according to the calendar period they were accepted for surgery: period 1—before 31 March 1997, and period 2—after that date. Decrease from 3 to 2.6 days per week in operating time, because of decrease in the number of anaesthetists in period 2, have raised a concern about differences in access to elective surgery between the periods.


All 1173 patients accepted for elective vascular surgery between 1 July 1994 and 31 March 1999, were eligible for the study. Table 1describes the distribution of cases by gender, age, urgency status, and period of acceptance on the waiting list. The follow up ended one year after the last patients were added to the waiting list. Altogether 1148 patients (97.9%) have left the waiting list within 40 weeks of enrolment. The remainder had waiting times between one and two years: 10 had surgery deferred indefinitely, including seven deferred by the surgeon, and three by the patient; planned surgery was declined by nine patients; and six patients, who have been initially deferred by the surgeon, eventually had surgery. Missing clinical records made it impossible to determine whether these unusually long waits were a result of the waiting list management. Therefore, we restricted our analysis to the first 40 weeks, and considered these 25 observations censored at 40 weeks.

Table 1

Demographic features, case mix characteristics, and waiting outcomes of 1173 patients enrolled on the waiting list

Table 1 also shows frequencies in the categories of waiting outcome. A total of 1063 patients (90.6%) underwent surgery, 25 (2.1%) were still waiting at the study end, and 85 (7.3%) were removed from the list without surgery for a variety of reasons: the patient's condition improved (22 patients), death occurred while the patient was awaiting surgery (4), the surgical risk became too great (41), or the patient decided against surgery (18). The proportions of those who received surgery within and beyond recommended time were similar (43.7% and 46.9% respectively). Among those removed without surgery, 27.1% was removed before recommended times expired. Altogether 431 patients (36.7%) waited longer than recommended times attributable to queue on the waiting list. Of 169 patients with scheduling delays, 28 (46.7%) were attributed to patients, 13 (21.7%) to hospital, and 19 (31.7%) to surgeons in period 1; and 45 (41.3%), 51 (46.8%), and 13 (11.9%), respectively, in period 2.


The waiting times were analysed as prospectively collected observations. The cumulative probability of admission as a function of time spent on the waiting list was estimated by using the product-limit method.5 All cases removed from the waiting list without surgery were treated as censored observations.6 By using the log rank and Wilcoxon tests, the likelihood of admission was compared by enrolment period.7 The occurrence of scheduling delays was modelled using a Cox regression model with a time dependent indicator variable that changes from 0 to 1 at the time of delay.2 The effect size was measured by relative rate of admission, derived from the Cox regression model in which we stratified on surgical procedure and age group. Gender and urgency status was entered as independent variables in multivariate regression to assess adjusted effects. The maximum recommended waiting time, measured in weeks, was used as a proxy for the time of delay in scheduling surgery. To calculate the weekly admission rate—that is, the average number of admissions from the waiting list—we divided the number of admissions by total number of patient-weeks on the list.


key points
  • Patients experiencing a delay in scheduling surgery have a lower admission rate after the event.

  • In between group comparisons, observed differences in waiting times may be because of different rates by which delayed patients are selected from the waiting list, whereas before a scheduling delay rates are the same in both groups.

The average number (standard error) of admissions was 10.6 (0.5) per 100 patients per week in period 1 and 9.9 (0.4) in period 2. Figure1 shows the estimated probabilities of elective admission versus waiting time among patients accepted for treatment during different periods. The difference between the estimates was statistically significant as measured by the Wilcoxon test (p<0.05), with the median time to admission 6 (95% confidence interval (CI)= 4, 6) weeks and 7 (95%CI= 6, 7) weeks, in period 1 and 2 respectively. However, when measured by the log rank test, the observed difference was non-significant, suggesting that the difference between periods was not constant over time. The weekly admission rate after a scheduling delay was 9.3 (1.3) in period 1 and 7.4(0.8) in period 2.

Figure 1

Estimated probabilities of elective admission by enrolment period.

Table 2 shows the relations between enrolment period and admission rate as measured by relative rates (RR). First, the period effect was estimated without taking intermediate events into account. After adjustment for gender, age, urgency status and surgical procedure, a lower admission rate was associated with period 2 (column 1). Those enrolled in period 2 had a 20% lower admission rate than patients accepted in period 1, RR = 0.8 (95%CI=0.7, 0.9). Second, the occurrence and timing of delays in scheduling operation were entered into the analysis. Column 2 shows that difference between the periods became smaller and only marginally significant after adjustment for the occurrence of scheduling delays, RR = 0.9 (95%CI=0.8, 1.0). Column 3 provides relative rates associated with scheduling delays for each enrolment period. The estimates were derived from the Cox regression model with three time dependent 0/1 variables coded in such a way that the before-delay admission rate in period 1 was the referent one. Variable 1 was 1 before delay in period 2; variable 2 was 1 after delay in period 1; and variable 3 was 1 after delay in period 2, otherwise zero.

Table 2

Estimates of relative rates2-150associated with enrolment period before (Model I) and after (Model II and III) adjustment for the occurrence of scheduling delays

Before a delay occurred, there were no differences in admission rate between the periods, RR = 0.9 (95%CI=0.8, 1.1). A lower admission rate was associated with the delay occurrence for both periods after adjustment for gender, age, urgency status and surgical procedure. In delayed patients, those enrolled in period 1 and period 2 had admission rate 40% and 60% lower than the period 1 patients admitted without scheduling delays, RR=0.6 (95%CI=0.4, 0.8) for period 1 and RR=0.4 (95%CI=0.3, 0.5) for period 2. The local test of no difference between these two relative rates has χ2=3.51 with one degree of freedom, suggesting marginal significance of the difference (p = 0.06).


Recently, several publications have examined methodological aspects of waiting time studies: the effect of excluding incomplete observations and competing events in calculating likelihood of admission8 or access probabilities,9 and selective admission bias.10 In this paper, we aimed to report the effect of exclusion of data on intermediate events in waiting time studies that compare differences between groups of patients. Our results provide evidence that patients experiencing a delay in scheduling operation have a lower admission rate after the event. The results show, furthermore, that observed differences in time to elective admission between two enrolment periods might be attributed to different rates by which delayed patients are selected from the waiting list, whereas there were not differences between periods before a scheduling delay happened.

Analysis of intermediate events raises some methodological complications. For instance, directly comparing patients who had a scheduling delay with patients who had not may produce misleading results. Firstly, patients from the first group must have stayed on a waiting list at least until the time of an event, while the second group might leave the list before it. Secondly, unlike main events, an intermediate event is not certain to occur; therefore, it is not appropriate to use the main event to censor the intermediate event for patients admitted without present the intermediate event.11 On the other hand, measuring time to admission from the date of an intermediate event introduces bias toward a higher rate in estimated probabilities of admission within a short time of enrolment.12 From a methodological perspective, analysis of time to admission is similar to analysis of heart transplant and abdominal aortic aneurysm survival data, or to studying incidence of postoperative infection. In heart transplant data, selective mortality before the operation may distort the estimate of the effect of receiving heart transplant.13 In aneurysm survival data, the mortality analysis by an intention to treat, elective aneurysm repair versus ultrasound surveillance, would probably produce misleading results as most of the surveillance patients also underwent the same surgery during the course of the trial, mostly attributable to an increased risk of aneurysm rupture.14 In postoperative infection study, selective discharge process, affecting outcome in the same direction, would probably overestimate the protective effect of discharge from hospital.15

To study the occurrence of intermediate events jointly with times to the main event of interest, Mantel and Byar (1974) proposed to put patients in the risk set of the “main events” group only after the intermediate event occurred.13 Crowley and Hu (1977) have suggested using time dependent covariate to model the occurrence and timing of an intermediate event that precedes the main event.16 Score tests were derived for between group comparison of survival times when an intermediate event may actually change survival.11 Such analysis proceeds conditionally on the times of intermediate events, and therefore precludes inferences based on the joint distribution of main and intermediate events. However, other indicators may help strengthen the inferences on how the intermediate event affects the admission rate. These include the probability of intermediate event as a function of time among those who experienced the event, and the probability that time after the intermediate event exceeds some stated value. Computation of these probabilities requires the joint distribution of time to main and intermediate events.17

Our analysis lacks data on coexisting diseases and socioeconomic status. In general, a large number of comorbid conditions at acceptance prevents an aggressive treatment. Therefore, given its association with some reasons for delay, comorbidity is a potential confounding factor for the observed association between delays in services scheduling and admission rate. Social class differences in health care access could be also relevant to exploring the way in which the events preceding admission affect admission rate,18 for example, through the hospital affiliation of referring physicians,19 or the preferential allocation of scarce resources.20

The results of the paper have implications for performance and policy evaluation across providers, periods, specialist remuneration and waiting list practices when time to elective admission is the main concern. They suggest that, in addition to adjustment for differences in needs composition, clinical and comorbid status at acceptance, waiting time studies that compare differences between group of patients require an adjustment for events preceding admission.


View Abstract


  • Funding: partial salary support for Dr Sobolev came from the Institute for Clinical Evaluative Sciences, Toronto, Canada.

  • Conflicts of interest: none.