Evidence-Based
Medicine into Practice

"Analysis conducted on the OR Black Box dataset analyzed 132 elective laparoscopic general surgeries and reported a median of 138 auditory distractions, 20 surgical errors, and 8 adverse events per case, and at least 1 cognitive distraction in 84 of the observed cases. Surgical errors and adverse events were found to be more common during dissection and reconstruction steps suggesting that some procedural steps may be more critical and require surgeon’s focused attention."

“There is currently no tool for assessing the teamwork performance of teams in crisis situations which was studied in clinical or simulated intraoperative crisis situations. This is a significant gap since most adverse events experienced by hospitalised patients involve surgery, with ineffective teamwork involved in a large proportion of cases”

“Experiential learning in the OR could be enhanced by a structured approach including objective assessment, structured debriefing, feedback, behavior modeling, and guided self-reflection. […] A coach can follow a structured methodology to guide the coachee to achieve self-determined goals. […] This methodology not only resulted in improved technical skill levels and reduction of errors but also enhanced the participants' self-assessment skills.”

“A significant improvement of residents’ nontechnical performance was observed after a single session of individual debriefing and feedback. Moreover, the intervention was well received by trainees, who felt both formal and informal feedback and debriefing on nontechnical aspects of operative performance should be integrated in surgical residency training.”

“Some non-technical attributes (e.g. risk attitude) change progressively throughout professional training and that trainees may grow towards a common profile set by staff members. […] A surgical ‘gold standard’ may be enriched with other non-technical attributes, to obtain a more complete surgical profile, which could prove useful as a benchmark in surgical candidate selection”

“Although quantifying teamwork can be challenging based on its complexity, there are five core characteristics that are purported to distinguish high-performing from low-performing teams: team leadership, adaptability, mutual performance monitoring, backup behavior, and team orientation”

“Communication failures are often at the root of adverse events for surgical patients; however, evidence to inform best communication practice in the operating room is relatively limited. […] The Operating Room Black Box is an appealing solution to the challenge of comprehensive communication assessment.”

“Assessment of technical performance needs to be done in a direct manner, rather than using surrogate variables. This will allow a better understanding of the technical deficiencies and development of targeted educational intervention designed to address these deficiencies.”

“This study is the first to show differences between low and high performers of laparoscopic tasks at the brain level. This pilot study has shown the feasibility of using fMRI to examine laparoscopic surgical skills.”

“This study is the first to show differences between low and high performers of laparoscopic tasks at the brain level. This pilot study has shown the feasibility of using fMRI to examine laparoscopic surgical skills.”

“3-D technology demonstrated a statistically significant reduction in errors and events and an improvement in OSATS scores in Laparoscopic Roux-en-Y gastric bypass procedures compared with 2-D systems”

“This study has provided the first evidence that surgeon performance in RARP is associated with early return to continence. Assessment of technical performance by a trained analyst using a valid metric of surgeon performance may be an independent predictor of patient outcomes in RARP.”

“These results support integration of endoscopy simulation into a structured curriculum incorporating instructional feedback and complementary didactic knowledge as a means to augment technical, cognitive, and integrative skills acquisition, as compared with SRL on virtual reality simulators.”

“NOTSS is particularly suited for investigators who aim to provide ratings for surgeons and trainees with different levels of nontechnical skills and to track changes in the level of performance over a period of time, especially when educational interventions are involved. Thus, there is evidence for NOTSS to serve as a discriminative or evaluative instrument.”

“Surgical operations are performed by a coordinated team of surgeons and surgical trainees, and consequently, assessment of nontechnical skills may benefit from holistic assessment of the collective surgical team. We found that the NOTSS system, although developed for assessment of individual surgeons, was a useful tool for observing and rating the surgical team.”

"Nontechnical skill assessment tools specific to the robotic surgical ecosystem must be developed and studied across various contexts of surgical training. […] Nontechnical performance should be assessed in conjunction with technical ability as a comprehensive measure of robotic skill acquisition, as they may be independent contributors to end outcomes.”

"In the era of Competence by Design medical education, we believe that automated methods represent the next step toward objective analysis of surgical trainees. Applications of these tools range from low-stakes assessments for trainee learning, to summative assessment at the surgical licensing examination level and beyond.”

“A curriculum of deliberate individualized practice on a VR simulator improves technical performance in the OR. This has implications to greatly improve the feasibility of implementing simulation-based curricula in residency training programs”

“Residents who participated in the curriculum surmounted the early portion of the learning curve during ex vivo training and outperformed conventionally trained residents in both the OR and during a simulated crisis scenario.”

“[…] The long-term objective of this project is to create a validated evaluation tool that will be used in residency training programs, as well as for credentialing practicing surgeons.”

“Communication and teamwork are intimately linked to device related interruptions. Communication failures occur often in the OR and are known to contribute to adverse event.”

“Credible and reliable technical and nontechnical performance standards can be set in General Surgery for the laparoscopic cholecystectomy […] with a real need to implement summative assessments that will determine whether appropriate levels of performance have been reached”

“[…] studies should focus on using objective data, such as derived by ORBB, to evaluate human factor behavior in the OR, and to define what type of human factors are most relevant and valuable to surgical safety, and to incorporate in team-based training.”

“Technical ability of bedside assistants is important to maximize surgeon performance, a finding that could potentially translate into better outcomes and fewer complications for patients.”

“Our data conclusively show that the surgeon and the fellow were unable to remember pertinent details from operations as few as 7 days earlier, despite reporting immediately postoperatively being mentally prepared and being allowed to adequately concentrate during the operation.”

“Debriefing that includes a systematic approach to crisis after simulated, postoperative crisis situations may decrease mental workload. This spare mental capacity may help improve trainees' clinical decision-making, reduce human errors, and/or direct attention toward acquiring technical/operative skills.”

“Error awareness is essential in daily surgical practice and surgical training. The current study highlights the benefits of detailed video analysis to create a database of common injury mechanisms and video clip repository that can be used in tailoring future training interventions.”

“While most stakeholders were supportive of the OR Black Box,[…] It is critical to ensure all stakeholders have adequate and accurate information about the OR Black Box system and research goals, and that the OR Black Box is positioned as a patient safety initiative.”

“The future of patient safety will be driven by interventions that harness technology to recognize patterns of performance or error that lead to patient harm and provide feedback to surgical teams that can be used to enhance future performance.”

“We developed and evaluated a novel instrument to measure severity of intraoperative events using video data from laparoscopic gastrointestinal operations. We underwent a robust development phase, which included reviews of the literature and video clips to generate a taxonomy of intraoperative events.”

“[…] The “sterile cockpit” concept utilized in aviation, in which non-essential communication is prohibited during critical phases of flight, is an example of an implementable intervention that may be applied to surgery to lessen the impact of perceived distraction”

“There is a lack of general agreement on how intraoperative adverse events (iAE) should be measured and reported. Our findings suggest that direct observation method has the most potential to identify and characterize iAE in detail.”

“Self-debriefing was cost-effective in crisis resource management simulation training when compared to instructor debriefing. This study provides evidence regarding cost-effectiveness that […] may help to optimize resource allocation in education.”

“The Fundamentals of Orthopaedic Surgery (FORS) simulator was developed to help increase the practice of relevant orthopaedic tasks by junior residents in a cost-effective manner and thereby allow universal access to all residents.”

“The authors developed a simple, low-cost pyeloplasty model that is highly acceptable to trainees and faculty and demonstrates preliminary content and construct validity.”

“[…] we have developed a video-based curriculum for perioperative nurses based on patient safety themes such as distractions and communication. Increased awareness of factors affecting patient care has the potential to inform best practices and therefore optimize patient safety.”

“As long as technology such as the Operating Room Black Box continues to be implemented by and for frontline health care providers in partnership with patients, it is likely to result in innovative quality improvement...”

“In laparoscopy, the Operating Room Black Box® (ORBB) […] may also enhance endovascular surgical practice; however, the use of a C-arm and X-rays pose important challenges...”

“Unfiltered surgical data would be a boon to those of us studying the OR. […] We will be able to systematically assess new techniques and technologies, identify best practices among numerous individual preferences, and provide evidence-based recommendations...”

“Unfiltered surgical data would be a boon to those of us studying the OR. […] We will be able to systematically assess new techniques and technologies, identify best practices among numerous individual preferences, and provide evidence-based recommendations...”

“Explainable artificial intelligence could be a powerful tool for intraoperative decision support, used in warning systems to help clinicians predict and avoid adverse events that may lead to complications.”

“Our proposed neural network models inferred temporal patterns from surgical instrument motions and associated them with surgical gestures, actions, and performance-related cues given validated rating scales.”

“The participants indicated that Marion Surgical K181 [virtual reality surgical sumulator] is an effective tool for surgery rehearsal, hand-eye coordination training, and that the simulator provides benefits over traditional training methods.”

“Although there are limited data on such a technology in the trauma bay, the Operating Room (OR) Black Box (Surgical Safety Technologies) has proven successful in characterizing adverse events and identifying latent safety threats in the OR.”

“Within-team debriefing may be as effective as instructor-led team debriefing for team-based simulation, which could improve cost-effectiveness and flexibility of scheduling of team-based simulation sessions.”

“The present study demonstrates that first year residents with laboratory bench model simulation training may technically perform advanced laparoscopic procedures in the OR.”

“There were significant differences for task time, movements, and errors when comparing novices and more experience surgeons training on a simulator module for laparoscopic appendectomy […].”

“resident level of training and laparoscopic experience correlated with technical performance during a simulation-based laparoscopic IVC injury crisis management scenario”

“As the impact of assessment scores on clinical outcomes becomes more evident across multiple surgical specialties, acquiring consequences evidence will become central when designing an assessment for trainee technical or non-technical performance”

“We report the first application of a progressive learning strategy to teaching colonoscopy. To our knowledge, this is the first comparative-effectiveness study to evaluate 2 different simulation curricula, with outcomes measured within the clinical setting.”

“Non-technical skills training can improve surgical performance in the simulated setting, lead to better attitudes towards patient safety, and promote more self-reflection among trainees”

“With a dramatic reduction in operative exposure, and in some cases, with a mandate to remain at home, how will surgical trainees maintain their surgical skill and intraoperative knowledge?”

“Technical skills learned on low-fidelity benchtop models, video-based trainers and virtual-reality models are transferable to the operating room.”

“Virtual-reality simulation allows for a more flexible environment than can be created by low- and highfidelity bench-top models. Many of the current VR systems allow for practice at varying levels of difficulty and across a wide range of clinical scenarios, thus accommodating learners at many levels.”

“The use of summative simulation-based assessments requires development of appropriate assessment models and tools, training of the faculty, and standard setting.”

“This prospective single-blinded randomized controlled trial demonstrated that completion of a comprehensive simulation-enhanced training curriculum for an advanced minimally invasive procedure resulted in superior psychomotor, and nontechnical skills, compared with conventional surgery training.”