SAFETY CULTURE
Nontechnical Skill Assessment of the Collective Surgical Team Using the NOTSS System
Published on
December 1, 2020
Annals of Surgery
Overview
The study aimed to validate the Non-Technical Skills for Surgeons (NOTSS) system for assessing the non-technical skills of surgical teams in real operating room (OR) settings. Traditionally used for individual assessment, the NOTSS system was applied to evaluate collective performance by analyzing ten 20-minute recordings of surgical teams at work. Five trained assessors rated the performances using the NOTSS framework, comparing their evaluations to reference ratings from a multidisciplinary expert committee.
Results indicated that the NOTSS system demonstrated adequate sensitivity and good inter-rater reliability, particularly in ratings of team performance and attending surgeons. A strong positive correlation was found between team and surgeon ratings, supporting concurrent construct validity. Participants reported that using the NOTSS system for assessing surgical teams was both acceptable and feasible. Overall, the findings suggest that the NOTSS framework can effectively evaluate non-technical skills in surgical teams, highlighting its potential for broader application beyond individual assessments.
Results
There was adequate system sensitivity when comparing participants' and reference ratings. Inter-rater reliability among the participants' ratings was good except for decision-making category. The level of inter-rater reliability was similar when rating teams and attending surgeons. There was strong positive correlation between teams' and attending surgeons' NOTSS ratings at category [Pearson coefficient 0.86, 95% confidence interval (CI) 0.82-0.89] and element levels (0.83, 95% CI 0.80-0.85), demonstrating evidence of concurrent construct validity. The participants felt that the use of NOTSS system to measure teams' nontechnical skills was acceptable and feasible to a fair extent.
