College admits that 10 vascular surgeons need to have their mortality rates reviewed

Ten vascular surgeons have been asked to check their performance data after analysis showed their mortality rates were higher than expected.

The 10 were among around 450 surgeons whose mortality rates for operations such as repair of aortic aneurysms went public at the beginning of July.1 At the time the Royal College of Surgeons said that all surgeons were operating “within the range expected.” But the BBC subsequently reported that the results of 10 surgeons were sufficiently poor to raise an alert.2

The protocol for the publication of the data says that risk-adjusted mortality rates of more than two standard deviations from the mean generate an “alert” and more than three an “alarm.”3 When an alert is raised, the surgical unit and the trust are asked to review the data and submit more complete data to the National Vascular Registry, if required.

David Cromwell, of the Clinical Effectiveness Unit at the Royal College of Surgeons, told the BMJ that the 10 surgeons fell into the first of these categories. “Their mortality rates were more than two standard deviations from the mean, but one would expect about one result in 40 to have this size of difference simply by chance. And since the published data cover more than 400 surgeons, the fact that 10 fall into this category is consistent with our expectations.”

Only if the alarm level is reached is there any question of stopping a surgeon operating. The registry expects alarms to be infrequent events and the first release of data triggered none, which is why the college said at the time that no surgeons were outliers. However, Cromwell admitted that the policy was “slightly ambiguous” since section 4 of the document explaining it says that surgeons falling into either the alert or the alarm category are outliers.

Cromwell told the BMJ that that section is about identifying potential outliers, and that section 6 of the same document spells out the process by which these cases are further investigated before they can formally be named as outliers.

“The distinction between potential and actual outliers is important,” he said. “It is essential for a clinical audit that a process of review is undertaken to ensure the figures are based on accurate data. It has been my experience that most units flagged as potential outliers have errors in their submitted data, and when corrected, have proven to have no case to answer.” He said that the college would be revising the document to make its policy clear.