Introduction

In 1995, the then chief medical officer, Sir Kenneth Calman, made recommendations for improving cancer care in England and Wales.1 Care was to be focused in a network of cancer units and centres that would have sufficient workload to permit appropriate subspecialisation and the development of expertise.

Those charged with implementing this recommendation have been exercised by the putative association between patient outcome and the number of procedures undertaken, whether by surgeon or by hospital. The Royal College of Surgeons has stated only that all patients requiring surgical management should be treated by surgeons with appropriate training and experience.2

The Clinical Outcomes Group recently stated that the quality of colorectal surgery can have a strong influence on patient survival.3 The Centre for Reviews and Dissemination further advised that concentrating surgery in the hands of those with better results could improve survival.4 Regardless of the reason, surgeon variability may influence outcome by as much as 20%.5 Evidence on the subject is inconclusive for several reasons, but “if there is no relationship, then the whole process of training and accreditation is called into question.”6

Detecting a relation often depends on the range over which observations are made, and commissioners of care may reasonably ask whether there is any relation locally between volume and outcome. Equally importantly, it should be asked whether surgical audit can reliably distinguish the “good achievers” from the rest and thus influence the shape of future services.

Our study attempts to address these issues, in particular whether patient mortality is related to the number of cases for which a surgeon or hospital is responsible.

Subjects and methods

Colorectal cancer register —The Northern Ireland colorectal cancer register was established in 1990 with the support of a local cancer charity. Although it produces an independent annual report, it now works more closely with the Northern Ireland cancer register. Clinical and pathological details of colorectal cancers diagnosed in Northern Ireland residents are collected from all hospitals and pathology laboratories serving the provinces (1.6 million population). Although we have previously reported incidence data, 7 8 this report is based on follow up (to the end of 1996) of the first 5 year cohort diagnosed with colorectal cancer between 1990 and 1994.

Subjects —Data recorded on each patient included the age, sex, and address of the patient, the site and Duke's stage of the tumour, the nature of surgery undertaken (emergency or elective and curative intent or palliative intent), and the consultant surgeon and hospital in charge of the case. The address of the patient was mapped to the electoral ward. The province's 566 wards were divided into fifths on the basis of their Townsend score, allowing each to be categorised according to a measure of material deprivation.

Follow up —Each year a follow up questionnaire is dispatched to the consultant in charge and cause of death is ascertained from the registrar general for all deceased patients. Our report deals with follow up status of patients at 31 December 1996. The median follow up period for our cohort was 53.8 months.

Results

Of the 3217 new patients registered for the period, 82 (2.5%)either had a biopsy only or there was insufficient information to determine the procedure, and these patients were excluded from the survival analysis.

Before 31 December 1996, 1512 patients (48.2%) had died; table 1 shows their characteristics. In 340 cases (10.8%) inadequate clinical information was retrieved for Duke's stage to be assigned, and in 183 cases (5.8%) insufficient address information precluded postcoding (and Townsend scoring).

In univariate analysis, significant predictors of survival at 2 years were age, Duke's stage, the degree of tumour differentiation, whether the liver was deemed clear by the surgeon, and the type of surgery (table 2). As we observed no significant effect on survival for tumour site, we combined the sites for further analysis.

After adjustment for case mix in a single level logistic regression analysis, there was only weak evidence of surgeon specific heterogeneity in patient survival (χ²=76.1; df=58; P=0.06). Likewise, there was only weak evidence of significant hospital specific heterogeneity in patient outcome. (χ²=27.1; df=18; P=0.08). We found no detectable effect of consultant workload or “experience” on outcome. Table 3 shows the relative odds of death at 2 years derived from the standard and multilevel logistic regression models and the relative hazards from a Cox's proportional hazards model. The base model contains the biological predictors before we simultaneously forced in the hospital and clinician variables.

In both the standard and multilevel models, the hospitals with higher caseloads—particularly those dealing with between 33 and 54 cases per year—had a slightly worse survival.

Using the entire 5 year cohort of patients, we ranked the performance of the surgeons firstly on the basis of unadjusted 2 year survival and then on survival adjusted for all the significant case mix variables in the logistic model (figure). Using an excess mortality of about 15% (roughly equivalent to the 75th centile of the surgeon's distribution), we compared the ranking of surgeons over the first 3 years with that over the last 2 years. Fourteen surgeons would have failed the test if a health authority had designated its specialist colorectal cancer surgeons by selecting those who achieved a mortality within the acceptable limits for patients treated between 1990 and 1993. However, over the next 2 years, six of them would have become a “safe” pair of hands and 11 others would no longer satisfy the criterion.

Consultants ranked (unadjusted, or adjusted for case mix) by mortality of patients after 2 years

• Download figure
• Open in new tab
• Download powerpoint

Discussion

We have shown that mortality within 3-4 years of diagnosis of colorectal cancer is not associated with annual caseload for surgeons in Northern Ireland dealing with up to 34 cases per year. Only 44% of the cases in this region were managed by surgeons dealing with 15 or more cases per year. This probably differs little from elsewhere in the United Kingdom.

The register we used ascertains its cases mostly through the pathology laboratories, so even if cases given only a clinical diagnosis were missed, their exclusion would not have affected the conclusions for surgical performance.

Other studies have investigated intermediate outcomes,12 and although we have not directly studied the surgeons' technical craft, we speculated whether for patients with similar Duke's stage, the type of surgery differed among surgeons with high and low workloads. Indeed, there was no difference in the proportions of curative versus palliative intent or elective versus emergency procedures. The proportion of cases with insufficient data for staging was broadly comparable across workload categories and we do not think this has biased our conclusions. Imprecise case mix adjustment, particularly if it was differential, may have biased the expected mortality. The collection of good quality information by surgeons is thus an indirect measure of outcome quality.

Although McArdle and Hole showed some significant difference between surgeon variation in patients' survival,12 they did not specifically investigate volume effects. Initial analyses from the Trent and Wales audits given in the report of the Clinical Outcomes Group show no case mix adjusted survival advantage (at 2 years) for patients treated by self reported specialists. Differences have been shown, for example, between survival after treatment in teaching and non-teaching hospitals,13 but whether they originate from selection bias or workload is not known.

Significant effects of operator workload have been shown in substantial literature from the United States, but most of these studies have focused on colectomy rather than on colorectal cancer.14^–16 The necessity to adjust simultaneously for effects of both hospital and operator specific workload is increasingly acknowledged. Poor outcomes in a specific hospital may be the result of the good outcomes of one expert surgeon being swamped by the poor results of several “occasional” surgeons.

Present policy suggests that we use audit to identify the “safe pair of hands.” The practice of surgeons in some districts has already been curtailed on the basis of samples of less than a year's work.17 While audit can identify a learning curve for surgical performance,18 the definition of an expert is rather fragile. If we assumed that the minimum clinically important attrition of survival for any given surgeon was 10%, then at least 150 cases would be required for this to be reliably detected, with 80% power and an α error (one sided) of 0.05.19 The particular outcome event and how its risk varies over time has a bearing on the type of case mix adjustment required.20 Clearly, audit focusing on the relation between volume and survival will, in most instances, be misplaced.

We can offer no explanation for the slightly worse outcome in the hospitals with medium volume workload. The relation is not a simple linear one. In a preliminary analysis we found it sensitive to the number of categories used for grouping. The grade of the operator was unknown to us but under emerging arrangements for clinical governance we think it justifiable to assign outcomes to the consultant in charge. We do not regard the level of findings from our hospital as conclusive, and they may reflect inadequate adjustment for unspecified case mix variables.

Whatever is responsible for the heterogeneity in outcome, such variation is of importance in multicentre trials. The potential benefits of treatments being studied may be swamped by such background variability.21 For example, preoperative radiotherapy has been shown to reduce local recurrence by 20% in university hospitals and by 60% in non-university hospitals.22

Several lessons emerge from our study. Firstly, making survival data available to wider audiences to rank the performance of surgeons may be unwise. It will often not identify the real problems. Even for breast cancer the differences in survival of patients between surgeons with high caseload and low caseload is not related to the type of surgery.23 Rather, the outcomes from colorectal cancer may depend on the clinical organisation of the service, the specialist surgeon being one trained to work effectively in a team and committed to ongoing quality assurance. Bringing together teams of surgeons, oncologists, pathologists, and palliative care specialists inevitably requires a commissioner to consider issues of economy of scale.

For the time being we must recognise that the volume-outcome debate may be a diversion. The rationale for providing multidisciplinary care and the need to enhance the power of clinical trials already provide compelling arguments for centralisation of cancer care.