Intended for healthcare professionals

CCBYNC Open access
Research

Prospective validation of classification of intraoperative adverse events (ClassIntra): international, multicentre cohort study

BMJ 2020; 370 doi: https://doi.org/10.1136/bmj.m2917 (Published 25 August 2020) Cite this as: BMJ 2020;370:m2917
  1. Salome Dell-Kuster, consultant and associate professor1 2 3,
  2. Nuno V Gomes, specialist registrar and research fellow1 3,
  3. Larsa Gawria, research fellow2 4,
  4. Soheila Aghlmandi, biostatistician2,
  5. Maame Aduse-Poku, research practitioner5,
  6. Ian Bissett, professor6,
  7. Catherine Blanc, consultant7,
  8. Christian Brandt, consultant8,
  9. Richard B ten Broek, postdoctoral researcher4,
  10. Heinz R Bruppacher, consultant9,
  11. Cillian Clancy, specialist registrar10,
  12. Paolo Delrio, director11,
  13. Eloy Espin, clinical professor12,
  14. Konstantinos Galanos-Demiris, consultant13,
  15. I Ethem Gecim, professor14,
  16. Shahbaz Ghaffari, consultant15,
  17. Olivier Gié, specialist registrar16,
  18. Barbara Goebel, specialist registrar17,
  19. Dieter Hahnloser, associate professor16,
  20. Friedrich Herbst, professor15,
  21. Ioannidis Orestis, consultant13,
  22. Sonja Joller, consultant18,
  23. Soojin Kang, research nurse5,
  24. Rocio Martín, specialist registrar12,
  25. Johannes Mayr, clinical professor17,
  26. Sonja Meier, consultant5,
  27. Jothi Murugesan, clinical fellow19,
  28. Deirdre Nally, specialist registrar10,
  29. Menekse Ozcelik, consultant20,
  30. Ugo Pace, consultant11,
  31. Michael Passeri, consultant21,
  32. Simone Rabanser, specialist registrar22,
  33. Barbara Ranter, consultant23,
  34. Daniela Rega, consultant11,
  35. Paul F Ridgway, professor10,
  36. Camiel Rosman, professor4,
  37. Roger Schmid, consultant24,
  38. Philippe Schumacher, head of department8,
  39. Alejandro Solis-Pena, consultant12,
  40. Laura Villarino, consultant25,
  41. Dionisios Vrochides, professor21,
  42. Alexander Engel, professor19,
  43. Greg O’Grady, associate professor6,
  44. Benjamin Loveday, consultant6,
  45. Luzius A Steiner, chairman1 3,
  46. Harry Van Goor, professor4,
  47. Heiner C Bucher, professor2 3,
  48. Pierre-Alain Clavien, chairman26,
  49. Philipp Kirchhoff, consultant27,
  50. Rachel Rosenthal, adjunct professor28
  1. 1Department of Anaesthesia, Prehospital Emergency Medicine and Pain Therapy, University Hospital Basel, Spitalstrasse 21, CH-4031 Basel, Switzerland
  2. 2Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital and University of Basel, Basel, Switzerland
  3. 3Department of Clinical Research, University of Basel, Basel, Switzerland
  4. 4Department of Surgery, Radboud University Medical Centre, Nijmegen, Netherlands
  5. 5Department of Anaesthesiology, Guy’s and St Thomas’ Hospital, London, UK
  6. 6Department of Surgery, University of Auckland, Auckland, New Zealand
  7. 7Department of Anaesthesiology, University Hospital Lausanne, University of Lausanne, Lausanne, Switzerland
  8. 8Department of Anaesthesiology, Bürgerspital Solothurn, Solothurn, Switzerland
  9. 9Department of Anaesthesiology, Schulthess Clinic, Zurich, Switzerland
  10. 10Department of Surgery, Tallaght University Hospital, Dublin, Ireland
  11. 11Colorectal Surgical Oncology, Abdominal Oncology Department, Istituto Nazionale per lo Studio e la Cura dei Tumori, “Fondazione Giovanni Pascale” IRCCS, Naples, Italy
  12. 12Colorectal Surgery Unit, Hospital Valle de Hebron, Autonomous University of Barcelona, Barcelona, Spain
  13. 13Fourth Surgical Department, G Papanikolaou Hospital, Aristotle University of Thessaloniki, Thessaloniki, Greece
  14. 14Department of Surgery, Ankara University Medical School, Ankara, Turkey
  15. 15Department of Surgery, Hospital of St John of God Vienna, Sigmund Freud University Vienna-Medical School, Vienna, Austria
  16. 16Department of Visceral Surgery, University Hospital Lausanne, University of Lausanne, Lausanne, Switzerland
  17. 17Department of Surgery, University Children’s Hospital Basel, Basel, Switzerland
  18. 18Department of Anaesthesiology, University Children’s Hospital Basel, Basel, Switzerland
  19. 19University of Sydney, Royal North Shore Hospital, Sydney, Australia
  20. 20Department of Anaesthesiology, Ankara University Medical School, Ankara, Turkey
  21. 21Department of Surgery, Carolinas Medical Centre, Charlotte, NC, USA
  22. 22Department of Anaesthesiology, Cantonal Hospital Graubünden, Chur, Switzerland
  23. 23Department of Vascular Surgery, Medical University of Innsbruck, Innsbruck, Austria
  24. 24Department of Surgery, Bürgerspital Solothurn, Solothurn, Switzerland
  25. 25Department of Anaesthesiology and Reanimation, Hospital Valle de Hebron, Autonomous University of Barcelona, Barcelona, Spain
  26. 26Department of Surgery and Transplantation, University Hospital Zurich, Zurich, Switzerland
  27. 27Department of General Surgery, University Hospital Basel, Basel, Switzerland
  28. 28Medical Faculty, University of Basel, Basel, Switzerland
  1. Correspondence to: S Dell-Kuster salome.dell-kuster{at}usb.ch
  • Accepted 23 June 2020

Abstract

Objective To prospectively assess the construct and criterion validity of ClassIntra version 1.0, a newly developed classification for assessing intraoperative adverse events.

Design International, multicentre cohort study.

Setting 18 secondary and tertiary centres from 12 countries in Europe, Oceania, and North America.

Participants The cohort study included a representative sample of 2520 patients in hospital having any type of surgery, followed up until discharge. A follow-up to assess mortality at 30 days was performed in 2372 patients (94%). A survey was sent to a representative sample of 163 surgeons and anaesthetists from participating centres.

Main outcome measures Intraoperative complications were assessed according to ClassIntra. Postoperative complications were assessed daily until discharge from hospital with the Clavien-Dindo classification. The primary endpoint was construct validity by investigating the risk adjusted association between the most severe intraoperative and postoperative complications, measured in a multivariable hierarchical proportional odds model. For criterion validity, inter-rater reliability was evaluated in a survey of 10 fictitious case scenarios describing intraoperative complications.

Results Of 2520 patients enrolled, 610 (24%) experienced at least one intraoperative adverse event and 838 (33%) at least one postoperative complication. Multivariable analysis showed a gradual increase in risk for a more severe postoperative complication with increasing grade of ClassIntra: ClassIntra grade I versus grade 0, odds ratio 0.99 (95% confidence interval 0.69 to 1.42); grade II versus grade 0, 1.39 (0.97 to 2.00); grade III versus grade 0, 2.62 (1.31 to 5.26); and grade IV versus grade 0, 3.81 (1.19 to 12.2). ClassIntra showed high criterion validity with an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91) in the survey (response rate 83%).

Conclusions ClassIntra is the first prospectively validated classification for assessing intraoperative adverse events in a standardised way, linking them to postoperative complications with the well established Clavien-Dindo classification. ClassIntra can be incorporated into routine practice in perioperative surgical safety checklists, or used as a monitoring and outcome reporting tool for different surgical disciplines. Future studies should investigate whether the tool is useful to stratify patients to the appropriate postoperative care, to enhance the quality of surgical interventions, and to improve long term outcomes of surgical patients.

Trial registration ClinicalTrials.gov NCT03009929.

Introduction

Over 310 million surgeries are conducted worldwide each year.1 One in six patients having elective surgery experiences at least one postoperative adverse event.2 The growing annual surgical volume13 and the increasing complexity of comorbid conditions in patients having surgery45 has caused a rise in adverse events related to surgery. Intraoperative adverse events are associated with adverse outcomes,678910 but in contrast with postoperative complications,111213 a prospectively validated instrument for standardised and transparent reporting of intraoperative adverse events is lacking. Uniform definitions of perioperative outcomes, however, are required for programmes to improve the safety of surgery, for benchmarking, and to adequately compare results from clinical research.14

In 2015, our group developed a CLASSification of Intraoperative Complications (ClassIntra version 1.0, formerly known as CLASSIC, supplementary table S1) in a Delphi process involving international, interdisciplinary experts, which was retrospectively validated.15 This tool defines intraoperative adverse events as any deviation from the ideal intraoperative course between skin incision and skin closure, and includes events related to surgery and anaesthesia. Similar to the widely used Clavien-Dindo classification for reporting postoperative complications (supplementary table S2),11 we sought to develop a generic classification, broadly applicable to all surgical disciplines, and including anaesthesia. To align with the validated Clavien-Dindo classification,1617 CLASSIC was updated by including five severity grades, maintaining the decision criterion on the need for treatment and severity of symptoms.

The aim of this international, multicentre, prospective cohort study was to investigate the validity and practicability of the updated classification in many different surgical disciplines.

Methods

Study oversight

This investigator initiated validation study of ClassIntra version 1.0 consisted of two parts. An international, multicentre, prospective cohort study was conducted in 18 centres in 12 countries in patients having any type of surgery requiring hospital admission for at least one night. The objective was to assess construct validity. Because no gold standard exists for the severity of intraoperative adverse events, we assessed the relation between the ClassIntra grades and other indirect measures that we expected to be related.18 In particular, we investigated the stepwise increasing effect of the severity grades of ClassIntra and relevant postoperative outcomes, such as postoperative complications, length of hospital stay, and length of surgery. The second part involved a survey of 10 fictitious case scenarios describing intraoperative adverse events. The survey was sent at the beginning of the prospective cohort study to physicians from the same international centres to grade severity. The aim was to assess criterion validity and reliability by comparing severity ratings by physicians with benchmark ratings set by the core team.

Across all centres, collaborators were trained in standardised data collection. Local study investigators confirmed the validity of the data from their centres. The results are reported according to STROBE (strengthening the reporting of observational studies in epidemiology).19

Patients and study procedures (prospective cohort study)

The centres prospectively included consecutive patients or a representative group of patients having any type of surgery from any surgical discipline. Excluded were day case surgeries, procedures not taking place in the operating room, procedures with no anaesthesia, and follow-up procedures.

The surgical and anaesthesia team prospectively recorded the severity grade and details of intraoperative adverse events that occurred between skin incision and skin closure, according to ClassIntra version 1.0 (table 1), during team sign-out20 at the end of surgery. Disagreement was resolved by consensus. All patients were followed up until discharged from hospital and were assessed daily by the postoperative care team to identify and grade postoperative complications according to the Clavien-Dindo classification.11 Because this was a pragmatic cohort study, performed during daily clinical practice, the postoperative team could not be blinded to the intraoperative adverse events, but was not aware of the ClassIntra grade. Postoperative complications after discharge, including mortality within 30 days, were assessed during structured follow-up visits or telephone calls to patients or their relatives.

Table 1

ClassIntra version 1.0 classification of intraoperative adverse events. The classification defines intraoperative adverse events as any deviation from the ideal intraoperative course occurring between skin incision and skin closure. Any event related to surgery and anaesthesia during the index surgery must be considered and should be rated directly after surgery.* A requirement is that the indication for surgery and the interventions conform to current guidelines

View this table:

Data describing the patients enrolled in the study were anonymised before entry into a secure web based electronic database. The database had automated checks for plausibility, consistency, and completeness of the data entries. Further consistency checks were performed after recruitment was finished.

ClassIntra

ClassIntra version 1.0 was derived from CLASSIC,15 which had four severity grades, but the same definition depending on the intervention required to treat the adverse event and the severity of symptoms (supplementary table S1). The only difference in the updated ClassIntra is that grade II has been divided into two grades. The additional grade (new grade III) describes a moderate deviation from the ideal course, which requires moderate treatment and results in severe symptoms. The rationale for updating the classification was to align with the number of grades in the Clavien-Dindo classification to avoid misclassification in grading intraoperative or postoperative death (supplementary table S2).11 A retrospective (unpublished) pilot study showed high reliability for ClassIntra.

We secured the rights to the ClassIntra classification by filing a trademark with the Swiss Federal Institute of Intellectual Property. The trademark was extended to relevant countries for our activities by international registration at the World Intellectual Property Organisation.

Outcomes

The primary outcome was the association of the most severe intraoperative adverse events, according to ClassIntra, with the most severe postoperative complications, according to the Clavien-Dindo classification.11 Secondary outcomes were the association of the most severe intraoperative adverse events with the sum of all postoperative complications, calculated by the comprehensive complication index,12 postoperative length of hospital stay, length of surgery, and 30 day mortality. Thirty day mortality was defined as death within 30 days of surgery, either in hospital or after discharge.

All models for the primary and secondary outcomes were adjusted for confounders: American Society of Anesthesiologists physical status,21 as a summary measure of perioperatively relevant comorbidities; age; length of surgery; complexity of surgery, graded as one of five categories according to the British United Provident Association (BUPA)2223; urgency of the procedure (planned or emergency); experience of the surgical and anaesthesia team (for a definition, see the statistical analysis plan in the supplementary file); and wound class.24 All confounders were defined according to biological plausibility and scientific rationale. Type of operation and complexity were recorded according to the BUPA classification.2223 BUPA categorises all commonly performed surgical procedures into five complexity grades (minor, intermediate, major, major plus, and complex major; supplementary table S15) and has been used previously for scientific purposes.2526

Inter-rater reliability and practicability of ClassIntra

An online anonymous survey (SurveyMonkey, San Mateo, CA, USA) of 10 fictitious case scenarios (supplementary table S3) describing intraoperative adverse events was sent to a representative sample of physicians from the participating centres. The scenarios were designed to cover a range of severity grades and medical specialties. The benchmark rating for each case scenario was determined by consensus of the core team. The physicians also rated the practicability of the instrument on a nine point scale, with end anchors of “very practical” and “not practical at all.”

Statistical analysis

A multivariable hierarchical proportional odds model was used to investigate the association between the most severe intraoperative and postoperative adverse event considering clustering within study centres. The proportional odds assumption was mainly assessed with the Brant test. Because of the sparseness of the data for the higher severity grades for intraoperative and postoperative adverse events, and breach of the proportionality assumption for the five categories of the Clavien-Dindo system, a three category classification (Clavien-Dindo grade 0, grades I-II, and grades III-V) was used. The rationale for these cut-offs was to distinguish an uneventful course from a minor or major deviation, and was in line with cut-offs used in most publications combining categories.16 The consistency of the coefficients was checked by performing an internal validation with bootstrapping methods, with 100 replications of samples from 2520 patients. A model with a binary endpoint (any v no postoperative complication) was used as a sensitivity analysis.

The association of the most severe intraoperative adverse event and the weighted sum of all postoperative complications (comprehensive complication index)12 was explored with a multivariable two level mixed effect regression model accounting for correlation between patients from the same study centre. Associations between the most severe intraoperative adverse event and the secondary endpoints of postoperative length of hospital stay and length of surgery were investigated in a multivariable two level mixed effect log linear regression model. The coefficients of the log linear regression models were back transformed to improve the interpretability of the results showing percentage changes27 in hospital days or duration of surgery for one unit increase in the corresponding covariate.

In the survey, the average agreement between the ratings of the physicians and the core team was calculated. Inter-rater reliability was assessed with the intraclass correlation coefficient based on the mean absolute agreement, estimated in a two way random effects model.28

The participating hospitals were a voluntary convenience sample. The planned sample size of 2500 patients allowed for robust estimation in the multivariable models, assuming at least one postoperative complication in 10% of the patients (that is, at least 250 events; for details, see the statistical analysis plan in the supplementary file).

Analyses were performed with Stata 14 (StataCorp, College Station, TX). Further details on data management and statistical analysis are in the statistical analysis plan (supplementary file).

Missing data

When an undefined complexity grade in the BUPA classification system (in 4% of procedures) was found, a grade corresponding to a similar procedure was used. This analysis was then compared with a complete case analysis. If no information was available on the presence of the anaesthesia resident or nurse, or whether they were in training or had graduated, we assumed that these team members were not present or were in training. There were no other missing values for outcomes or covariates, except for data on 30 day mortality in patients lost to follow-up.

ClassIntra study group

The ClassIntra study group consisted of a multidisciplinary, international team of anaesthetists, surgeons with a range of surgical specialties, epidemiologists, and statisticians with extensive expertise in perioperative care.

Patient and public involvement

This research was a multicentre collaboration to evaluate a newly developed classification for intraoperative adverse events. The data and software were not publicly available for analysis by patients or members of the public. Our protocol did not include patient and public involvement in the planning or implementation of this research. No patients were asked to advise on interpretation or writing up of the results. However, future research will evaluate the effect of intraoperative adverse events measured with ClassIntra on patient reported outcomes.

Results

Patient and procedural characteristics

Of 2640 patients screened, 2520 were enrolled in 18 secondary and tertiary centres in North America, Europe, and Oceania, representing 12 countries. All patients were recruited between 14 February 2017 and 27 July 2018. The median number of patients enrolled per site was 130 (interquartile range 100-176). The main reason for exclusion was day surgery (fig 1). Patients had a range of surgical procedures from different surgical disciplines, except ophthalmic surgery (table 2), with 44% (n=1090) of surgeries corresponding to the two highest complexity levels. Patient and procedural characteristics are described in table 2 (examples of the most frequent types of surgical procedures are in supplementary table S15). Baseline characteristics were similar in patients with and without intraoperative adverse events. The American Society of Anesthesiologists’ physical status classes, and the complexity of the surgical procedure, however, were higher in patients with an intraoperative adverse event.

Fig 1
Fig 1

Flow diagram of patients in the validation study of ClassIntra

Table 2

Patient and procedural characteristics for the whole study population and for subgroups with and with no intraoperative adverse events

View this table:

Intraoperative adverse events according to ClassIntra

Of 2520 patients enrolled, 610 (24%) experienced 778 intraoperative adverse events (table 3). The most severe intraoperative adverse event per patient was grade I in 161 patients (6.4%), grade II in 309 (12%), grade III in 122 (4.8%), and grade IV in 18 patients (0.7%). No intraoperative deaths occurred (table 3).

Table 3

Outcomes for the whole study population and for subgroups with and with no intraoperative adverse events

View this table:

Postoperative complications

During the postoperative course, 2509 complications were seen in 838 patients (33%). Table 3 gives details of the severity grades and number of postoperative complications.

Multivariable analysis showed a gradual increase in the odds of a more severe postoperative complication with an increasing grade of the most severe intraoperative adverse event (fig 2; supplementary table S5). Consistently, a stepwise increase in the sum of all postoperative complications (comprehensive complication index) with an increasing grade of ClassIntra was seen (supplementary table S6).

Fig 2
Fig 2

Multivariable hierarchical proportional odds model for the most severe postoperative complications according to the Clavien-Dindo classification (three categories for outcome: no postoperative complications, grades I-II, and grades III-V), adjusted for the most relevant confounders. ASA=American Society of Anesthesiologists’ physical status level

More secondary outcomes

Median postoperative length of hospital stay was 3 days (interquartile range 2-6) in all patients, and was doubled in patients with at least one intraoperative adverse event (median 6 days; interquartile range 3-9 days) compared with patients with no intraoperative adverse events (3; 1-5 days) (table 3). In the multivariable analysis, median postoperative length of hospital stay increased with each ClassIntra grade by 6.1% (95% confidence interval 2.8 to 9.3%; supplementary table S7).

The median length of surgery was 100 minutes (interquartile range 60-170) with a maximum length of 760 minutes (table 3). The length of surgery was longer in patients experiencing an intraoperative adverse event, with an increase of 16% in median length of surgery with each grade increase in ClassIntra (95% confidence interval 13% to 18%; supplementary table S8). Thirty day follow-up was available for 2372 (94%) patients (fig 1). Mortality at 30 days was 1.1% (n=26), with a higher incidence in patients with an intraoperative event (table 3 and table S9).

Survey

The response rate to the survey was 83% (136/163 physicians). About a third of respondents were anaesthetists and two thirds were surgeons from various surgical disciplines with a range of years’ experience (supplementary table S4). Reliability among physicians showed an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91). Overall agreement with the ratings of the core team was 61% (interquartile range 42-71%; supplementary figure S1), with deviation usually explained by physicians rating one grade lower than the core team. Physicians rated the practicability of ClassIntra as 6 out of 9 (interquartile range 5-7).

Discussion

Principal findings

This international multicentre study showed that ClassIntra is a valid and standardised tool for assessing intraoperative adverse events, with high construct validity, given the increasing effects of severity grades on important patient outcomes. ClassIntra also showed good criterion validity and practicability across different surgical disciplines.

The association between severe intraoperative and postoperative complications, well established in abdominal surgery,68 was replicated in a heterogeneous population of surgical patients from a range of surgical disciplines. The association of intraoperative adverse events and postoperative length of hospital stay might have been mediated by postoperative complications (that is, more postoperative complications resulted in a longer stay in hospital after operation). The association between intraoperative adverse events and length of surgery was most likely related (that is, more adverse events might have prolonged surgery, which in turn could have led to more intraoperative adverse events).

Strengths and limitations of the study

ClassIntra is a comprehensive classification, with a patient centred perspective, because it records any event related to anaesthesia or surgery, and grades the outcomes according to those relevant to the patient, the symptoms caused by the event, and the measures taken to treat the event. ClassIntra was derived from CLASSIC, which was developed by an interdisciplinary, international group of anaesthetists, surgeons, epidemiologists, and statisticians.15 ClassIntra aligns with the Clavien-Dindo classification of postoperative complications,11 allowing the reporting of intraoperative and postoperative adverse events.

Our robust study design, based on a large sample size involving a range of surgical disciplines, and on prospective multicentre data collection, implies good generalisability. Good generalisability always introduces some heterogeneity, which was accounted for by adjusting the analyses for the complexity of the surgical procedure and patient related factors. Whether the results are mainly driven by the largest subgroup of patients having an intra-abdominal procedure will be investigated in a secondary analysis comparing the results in several subgroups. The primary advantage of a broad classification is that it increases uniformity in reporting intraoperative adverse events and allows comparison of results between different centres and times. In our study, details of each intraoperative adverse event were collected. Our group is currently investigating how the additional information can be used to support quality improvement in the future. We therefore encourage adding details about the events and intervention specific intraoperative outcomes of interest to the ClassIntra grade, whenever possible within a framework of standardised core outcomes.29

To keep it practical and simple, the classification uses generic definitions of the five severity grades; individual judgment is necessary for interpretation. Similarly, defining an event as deviation from the ideal intraoperative course requires personal judgment. Inter-rater reliability in the survey was strong and disagreement was mostly by one grade. We found the inter-rater reliability results reassuring, given that the physicians had various backgrounds and were grading intraoperative adverse events in different surgical disciplines and in anaesthesia. In clinical practice and in our prospective cohort study, joint assessment by surgeons and anaesthetists during sign-out20 should lower this variability and the risk of under-reporting and downgrading of intraoperative adverse events. As clinicians become more familiar with ClassIntra, we believe the system will be easier to use, similar to the experience with the Clavien-Dindo classification after its introduction into routine clinical practice. Generally, different professional backgrounds, levels of experience, and cultural diversity among physicians might have explained the practicability ratings of less than five in 25% of the survey participants and ratings greater than seven in another 25%.

Because our project was conducted as part of daily clinical practice, having an independent person to record intraoperative and postoperative events would not have been feasible, and blinding of those assessing postoperative outcomes would have been unethical. The direction of the potential bias is unclear, however, as under-reporting and over-reporting could, theoretically, occur. Under-reporting and downgrading of complications is a problem in healthcare settings and should be handled with a positive, non-discriminative learning culture. A classification for complications should be sensitive enough to monitor and detect rating variability caused by downgrading.3031

Our secondary endpoints included length of hospital stay. We acknowledge that this outcome might be affected by the variables included in the model (that is, the most severe intraoperative adverse event and the confounding variables) and also by cultural norms, logistical issues, and economic considerations. This analysis should be viewed as supporting evidence as it showed consistent stepwise increasing association in alignment with all other postoperative endpoints.

Findings in relation to other studies

Other classification systems for intraoperative adverse events have been published.32 Kaafarani and colleagues8 classified intraoperative adverse events into six grades. Grades I-IV depend on the intervention required to treat the intraoperative adverse events and the deviation from the planned procedure. Grade V refers to an intraoperative injury missed during surgery and requiring re-operation within seven days. Grade VI refers to intraoperative death. This classification was based on a retrospective analysis of routinely collected data in selected patients with accidental puncture or laceration. Although their classification approach differs considerably from ClassIntra, the authors confirmed an association between major intraoperative and postoperative adverse events.8 Our study was based on prospective data in unselected patients from a range of surgical interventions and considered any event related to surgery or anaesthesia. We did not include re-operation because of the difficulty in determining if an intraoperative injury or a postoperative complication was responsible for the re-operation.

A classification of surgical intraoperative adverse events for laparoscopic surgery was proposed, with five grades similar to ClassIntra, but excluded adverse events related to anaesthesia.33 Others used readily available information, such as quality indicators of the intraoperative period, to predict postoperative outcome. Coulson and colleagues estimated the acute risk change in the probability of death before and after operation based on two validated risk scores.3435 The 10 point Apgar score for surgery was developed in patients having a colectomy and was evaluated in a broader surgical population.3637 Prediction of postoperative complications was based on estimated blood loss, lowest mean arterial pressure, and heart rate. Although both scores showed good predictive ability, they provided only a surrogate marker for adverse intraoperative events.

Bothner and colleagues developed an incidents, events, and complications scale related to anaesthesia, based on a set of 63 events with five ascending severity grades depending on the effect on postoperative care and outcome.38 The scale was implemented for quality assurance and covered the whole period of anaesthesia, from induction to discharge to the ward or to the intensive care unit. Their reported rate of 22% adverse events was relatively higher than our findings where events related to anaesthesia and surgery were taken together, which could partly be explained by the restricted period covered in our study.

Implications for clinicians or policy makers

Patient safety in the perioperative period has gained considerable interest and successful interventions, such as the reduction in morbidity and mortality after introduction of a surgical safety checklist, have been seen.20 ClassIntra covers a major unmet need, as the first prospectively validated classification system that is simple and generally applicable to all surgical disciplines, to our knowledge. It takes a holistic view by integrating any adverse event, regardless of origin. Also, because innovative surgical techniques are increasingly complex and implemented quickly, preventing morbidity during the implementation period is important.39 ClassIntra should be included in routine practice to alert the postoperative care team to patients with intraoperative adverse events and at risk of further postoperative complications.

Future research

We acknowledge that our definition of intraoperative adverse events included only the time between skin incision and closure. Induction of and emergence from anaesthesia were not covered, which can have risks for more adverse events. Assessing the construct validity of a new classification is an ongoing process, which requires confirmation by more convergent and discriminative validity assessments to increase credibility. In future research, we will evaluate the properties of ClassIntra extending the assessment period to include the periods related to anaesthesia (that is, induction and emergence), focusing on patient reported outcomes, thus expanding the hypothetical construct for assessing validity.

ClassIntra can be used in clinical research as a defined core outcomes measure as it provides standard definitions, allowing accurate communication of research results and comparisons across research studies.1140 ClassIntra can help evaluate the origin of the complications during the surgical procedure, providing more granularity to the complication profile for each procedure or surgical unit. Also, ClassIntra can be used in critical incidence reporting systems,41 for measuring individual performance in educational settings and system performance for benchmarking by institutions, and for outcome research.Future implementation studies should evaluate this tool in daily clinical practice for risk stratification of postoperative care or for planning resources and hospital capacities, in view of the increase in postoperative length of stay with the increasing severity of the ClassIntra grade. For day surgeries, the risk of perioperative complications is minimal. Future research should investigate whether assessing intraoperative complications is associated with early visits to the general practitioner or readmission.

Conclusion

ClassIntra version 1.0 is a validated and standardised tool to quantify and qualify intraoperative adverse events in clinical practice and research. It showed good reliability and practicability for all surgical disciplines and for anaesthesia. We reproduced the well established association between the severity grade for intraoperative adverse events and important postoperative outcomes. Hence ClassIntra is a useful tool for recording intraoperative adverse events and should be incorporated into the perioperative surgical safety checklist to highlight recurrent and potentially avoidable complications. ClassIntra could improve the technical aspects of intraoperative care and provide clarity in audits of practice.

What is already known on this topic

  • Intraoperative adverse events are a concern for patients, anaesthetists, and surgeons because of their association with adverse postoperative outcomes

  • In contrast with postoperative complications, prospectively validated systems for reporting intraoperative adverse events are lacking

  • Uniform definitions and transparent reporting of intraoperative adverse events are required for programmes to improve the safety of surgery and to compare results from clinical research

What this study adds

  • ClassIntra is the first prospectively validated classification system with good reliability and practicability that includes intraoperative adverse events related to surgery and anaesthesia

  • ClassIntra covers a major unmet need as it provides a standardised instrument to quantify and qualify intraoperative adverse events in clinical practice and clinical research for a range of surgical disciplines and for anaesthesia

Acknowledgments

We thank Ramon Saccilotto who programmed and maintained the online database; Allison Dwileski for editorial assistance on the manuscript; Christian Schindler for statistical advice; and all of the centres in this investigator initiated study for their support. In particular, we thank Esther Seeberger, Claudia Blaesi, Franziska Merz, Samuel Heim, and all participating surgical disciplines from University Hospital Basel, Basel, Switzerland; Martin Taube, Kata Papp, and Blanca Schuster, University Children’s Hospital Basel, Basel, Switzerland; Michael Winiker, University Hospital Lausanne, Lausanne, Switzerland; Victoria Koenig-Oberhuber and the preoperative ambulatory unit team, Cantonal Hospital Graubünden, Chur, Switzerland; David Iannitti and Hiral Amin, Carolinas Medical Centre, Charlotte, NC, USA; Deniz Yalman, Ankara University Medical School, Ankara, Turkey; Christos Chatzakis, Savvas Symeonidis, Natalia Antigoni Savvala, Nikolaos Antoniou, Dimitrios Konstantaras, Lydia Loutzidou, Loukiani Kitsikosta, Grigorios Gkasdaris, Stefanos Bitsianis, Ioannis Mantzoros, Manousos George Pramateftakis, Efstathios Kotidis, and Konstantinos Tsalis, medical school at Aristotle University of Thessaloniki, Greece; the surgical trials unit team at University of Auckland, Auckland, New Zealand; Olivia Kirkland, Royal North Shore Hospital, Sydney, Australia; Astrid Chiari and René Schmutz, Department of Anaesthesiology, and Bernhard Dauser and Tudor Birsan, Department of Surgery, Hospital of St John of God Vienna, Austria; and Kariem El-Boghdadly and Gill Arbane, Guy’s and St Thomas’ Hospital, London, UK.

Part of the preliminary findings of this research were presented at the following meetings: (11 November 2017) poster presentation at the Congress of the Swiss Society for Anaesthesiology and Resuscitation (Interlaken, Switzerland); (2 November 2018) poster presentation at the Congress of the German Society of Gynaecology and Obstetrics (Berlin, Germany); (6 February 2019) QuickShot presentation at the 14th Annual Academic Surgical Congress (Houston, TX, USA); (9 May 2019) short paper at the International Surgical Congress of the Associations of Surgeons of Great Britain and Ireland (Telford, UK); (9 May 2019) oral presentation in the general surgery section of the Royal Australasian College of Surgeons’ annual scientific congress (Bangkok, Thailand); (16 May 2019) oral presentation at the 106th Annual Meeting of the Swiss Society of Surgery (Bern, Switzerland), and abstract published as society abstract supplement (Br J Surg 2019;106(S4):9); (15 May 2019) oral presentation at the XXIII National Meeting of the Spanish Association of Coloproctology (Valladolid, Spain); (3 June 2019) oral presentation at the European Anaesthesiology Congress Euroanaesthesia of the European Society of Anaesthesiology (Vienna, Austria), and abstract published as society abstract supplement (Eur J Anaesthesiol 2019;36(S57):313); (26 June 2019) oral presentation at the Swiss Congress of Gynaecology (St Gallen, Switzerland); (15 August 2019) oral presentation at the World Congress of Surgery (Krakow, Poland); (26 September 2019) free paper presentation at the 14th scientific and annual meeting of the European Society of Coloproctology (Vienna, Austria); (7 November 2019) oral presentation at the Congress of the Swiss Society for Anaesthesiology and Resuscitation (Interlaken, Switzerland; prize for second best oral presentation); and (8 November 2019) oral presentation at the National Surgery Congress of Spain (Santander, Spain). The authors were awarded the first prize for Innovation Qualité by the Swiss Academy of Quality in Medicine (SAQM) of the Swiss Medical Association.

Footnotes

  • Contributors: SD-K and NVG contributed equally as first authors. PK and RR contributed equally as last authors. SD-K, NVG, and RR did the literature search. SD-K, NVG, PFR, HCB, P-AC, LAS, PK, and RR designed the study. S D-K, NVG, LG, RM, PFR, AE, GO, BL, HV, and PK collected the data. SD-K, LG, and SA analysed the data. SD-K, LG, SA, AE, GO, BL, HV, HCB, and RR interpreted the data. SD-K, NVG, RM, HCB, and RR wrote the manuscript. LG, SA, PFR, AE, GO, BL, HV, CR, P-AC, LAS, and PK provided critical feedback on the manuscript. MA-P, IB, C Bl, CBr, RTB, HRB, CC, PD, EE, KG-D, IEG, SG, OG, BG, DH, FH, OI, SJ, SK, JMa, SM, JMu, DN, MO, UP, MP, SR, BR, DR, CR, RS, PS, AS-P, LV, and DV collected the data and provided critical feedback on the manuscript.

  • All authors revised the manuscript for important intellectual content, approved the final version of the manuscript, and agreed to be accountable for all aspects of the work. SD-K is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: The research was supported by grants from the Goldschmidt-Jacobson Foundation, University of Basel, Foundation for Research and Education in Anaesthesiology, Switzerland, and VSB fonds (Verenigde Spaar Bankenfonds), Netherlands. The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all data.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Goldschmidt-Jacobson Foundation, University of Basel, Foundation for Research and Education in Anaesthesiology, Switzerland, and VSB fonds (Verenigde Spaar Bankenfonds) for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work. RR is an employee of F Hoffmann-La Roche (includes Stock-Settled Appreciation Rights). The study has no connection to her employment by the company. RR continues to be affiliated to the University of Basel.

  • Ethical approval: The study was approved by the regional ethics committee of Ethikkommission Nordwest- und Zentralschweiz (reference No Req-2016-00469) as Swiss lead ethics committee, and by the local ethics committees of all international centres. Participants gave oral or written informed consent compliant with the relevant legal requirements of each centre.

  • Data sharing: Anonymised patient level data are available for investigators whose proposed use of the data has been approved by a review committee identified for this purpose.

  • The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

  • Dissemination to participants and related patient and public communities: To reach wide dissemination, the results of the study were presented at many international meetings. After publication, the results will be disseminated to relevant medical societies and patient organisations. The use of an (adapted) version of our web based database could easily be broadly implemented in other hospitals other than the study sites, which are interested to use ClassIntra version 1.0 as a tool for internal quality control in daily clinical practice and in research.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

http://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References

View Abstract