Evidence-Based
Medicine into Practice

Last updated on: March 10th, 2023

Filter by category

Automated Capture of Intraoperative Adverse Events Using Artificial Intelligence: A Systematic Review and Meta-Analysis.

"There is a demonstrable need for the standardization of iAE identification and reporting in surgery, which may be improved with the incorporation of AI technology."

Analyzing interprofessional teamwork in the operating room: An exploratory observational study using conventional and alternative approaches.

"As healthcare organizations continue to prioritize teamwork improvement, SEIPS may provide valuable insights regarding teamwork behavior and the broader context influencing performance. This may ultimately facilitate the development of multi-level teamwork interventions and optimize their impact on both provider and patient outcomes."

Patient Safety in the Operating Room During Urologic Surgery: The OR Black Box Experience.

"Our data demonstrate that IAEs occur frequently during all phases of the operation at hand."

Patient Safety in the Operating Room During Urologic Surgery: The OR Black Box Experience.

"Our data demonstrate that IAEs occur frequently during all phases of the operation at hand."

Bringing the patient voice into the operating room: engaging patients in surgical safety research with the Operating Room Black Box®.

"The Operating Room Black Box® enhances surgical patient safety through transparent and proactive analysis of human factors to detect and prevent avoidable errors."

System Factors Affecting Intraoperative Risk and Resilience: Applying a Novel Integrated Approach to Study Surgical Performance and Patient Safety.

"Safety threats and resilience supports were found to be systematic in the surgical setting. Identified safety threats should be prioritized for remediation, and clinician behaviors that contribute to fostering resilience should be valued and protected."

Implementation of an Innovative Technology Called the OR Black Box: A Feasibility Study.

"Implementation and use of an innovative and comprehensive data capture platform like the OR Black Box in a gynecological setting was feasible. Stakeholders, ie, management, nearly all OR staff, and patients embraced the initiative. Continuous evaluation, information meetings, and adjustments were nonetheless required to sustain momentum."

Surgeons’ Leadership Style and Team Behavior in the Hybrid Operating Room: Prospective Cohort Study.

"Assess the relationship between surgeons’ leadership style and team behavior in the hybrid operating room through video coding."

Step, Error, and Event Frameworks in Endovascular Aortic Repair.

"A detailed evaluation tool for the procedural steps, errors, and events in infrarenal EVAR was developed. This tool will be validated on recorded procedures in future work: It may focus skill development on common errors and hazardous steps. This tool might be used to provide high-quality feedback on technical performance of trainees and experienced surgeons alike, thus promoting surgical mastery."

Electroencephalography can provide advance warning of technical errors during laparoscopic surgery.

"These results show that distinct neural signatures are predictive of technical error in laparoscopic surgery. If replicated with low false-alarm rates, an EEG-based EWS of technical errors could be used to improve individualized surgical training by flagging imminent unsafe actions—before errors occur and cause patient harm."

Creating a Practical Transformational Change Management Model for Novel Artificial Intelligence-Enabled Technology Implementation in the Operating Room.

"Key CM strategies and a new CM model addressing the unique needs and characteristics of operating room novel technology implementation were identified and created."

Surgical data science – from concepts toward clinical translation.

"While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field."

Analyzing and Discussing Human Factors Affecting Surgical Patient Safety Using Innovative Technology: Creating a Safer Operating Culture.

"Patient safety threats identified by the MDR and discussed by the operating room team were most frequently related to communication, teamwork, and situational awareness. To create an even safer operating culture, educational and quality improvement initiatives should aim at training the entire operating team, as it contributes to a shared mental model of relevant safety issues."

Patient Perceptions of Audio and Video Recording in the Operating Room.

"Patients have a diverse set of views about the potential benefits, risks, and uses for OR data and consider themselves to be important stakeholders. Our study identifies pathways and potential challenges to implementation of continuous audio/video recording in ORs."

Assessing endovascular team performances in a hybrid room using the Black Box system: a prospective cohort study.

"The ORBB facilitates holistic workplace-based assessment of endovascular performance in a HR by combining objective assessment parameters and rating scale-based evaluations. This technology has the potential to improve vascular surgical practice, though human input remains crucial."

Healthcare professionals’ perception of safety culture and the Operating Room Black Box technology before clinical implementation: a cross-sectional survey.

"The different healthcare professional groups had diverse perceptions about safety culture, but were mainly concerned about safety climate and teamwork in the OR. Impostor phenomenon decreased with age. All groups were unconcerned about digital information sharing."

The Operating Room Black Box: Understanding Adherence to Surgical Checklists.

"ORBB provides the unprecedented ability to assess not only compliance with surgical safety checklists but also engagement and quality. Utilization of this technology allows the assessment of compliance in near real time and to accurately address safety threats that may arise from noncompliance."

Perceptions of safety culture and recording in the operating room: understanding barriers to video data capture.

"The objective of this study was to assess pre-implementation attitudes of OR staff toward operative recording and explore the relationship of these attitudes to the themes of (1) safety culture, (2) impostor syndrome, and (3) privacy concerns."

The effect of intraoperative distractions on severe technical events in laparoscopic bariatric surgery.

"Analysis conducted on the OR Black Box dataset analyzed 132 elective laparoscopic general surgeries and reported a median of 138 auditory distractions, 20 surgical errors, and 8 adverse events per case, and at least 1 cognitive distraction in 84 of the observed cases. Surgical errors and adverse events were found to be more common during dissection and reconstruction steps suggesting that some procedural steps may be more critical and require surgeon’s focused attention."

Measuring the teamwork performance of teams in crisis situations.

“There is currently no tool for assessing the teamwork performance of teams in crisis situations which was studied in clinical or simulated intraoperative crisis situations. This is a significant gap since most adverse events experienced by hospitalised patients involve surgery, with ineffective teamwork involved in a large proportion of cases”

Comprehensive Surgical Coaching Enhances Surgical Skill in the Operating Room: A Randomized Controlled Trial.

“Experiential learning in the OR could be enhanced by a structured approach including objective assessment, structured debriefing, feedback, behavior modeling, and guided self-reflection. […] A coach can follow a structured methodology to guide the coachee to achieve self-determined goals. […] This methodology not only resulted in improved technical skill levels and reduction of errors but also enhanced the participants' self-assessment skills.”

Implementation of an Effective Strategy for Teaching Nontechnical Skills in the Operating Room: A Single-blinded Nonrandomized Trial.

“A significant improvement of residents’ nontechnical performance was observed after a single session of individual debriefing and feedback. Moreover, the intervention was well received by trainees, who felt both formal and informal feedback and debriefing on nontechnical aspects of operative performance should be integrated in surgical residency training.”

Non-technical attributes and surgical experience: A cross-sectional study comparing communication styles and attitudes in surgical staff, trainees and applicants

“Some non-technical attributes (e.g. risk attitude) change progressively throughout professional training and that trainees may grow towards a common profile set by staff members. […] A surgical ‘gold standard’ may be enriched with other non-technical attributes, to obtain a more complete surgical profile, which could prove useful as a benchmark in surgical candidate selection”

Measuring the teamwork performance of operating room teams: a systematic review of assessment tools and their measurement properties

“Although quantifying teamwork can be challenging based on its complexity, there are five core characteristics that are purported to distinguish high-performing from low-performing teams: team leadership, adaptability, mutual performance monitoring, backup behavior, and team orientation”

Interprofessional communication in the operating room: a narrative review to advance research and practice.

“Communication failures are often at the root of adverse events for surgical patients; however, evidence to inform best communication practice in the operating room is relatively limited. […] The Operating Room Black Box is an appealing solution to the challenge of comprehensive communication assessment.”

The Effect of Technical Performance on Patient Outcomes in Surgery: A Systematic Review.

“Assessment of technical performance needs to be done in a direct manner, rather than using surrogate variables. This will allow a better understanding of the technical deficiencies and development of targeted educational intervention designed to address these deficiencies.”

Brain activation during laparoscopic tasks in high- and low- performing medical students: a pilot fMRI study.

“This study is the first to show differences between low and high performers of laparoscopic tasks at the brain level. This pilot study has shown the feasibility of using fMRI to examine laparoscopic surgical skills.”

Experienced bedside-assistants improve operative outcomes for surgeons early in their learning curve for robot assisted laparoscopic radical prostatectomy.

“This study is the first to show differences between low and high performers of laparoscopic tasks at the brain level. This pilot study has shown the feasibility of using fMRI to examine laparoscopic surgical skills.”

Assessment of 3-Dimensional vs 2-Dimensional imaging and technical performance using a multiport intraoperative data capture and analytic System.

“3-D technology demonstrated a statistically significant reduction in errors and events and an improvement in OSATS scores in Laparoscopic Roux-en-Y gastric bypass procedures compared with 2-D systems”

Surgeon Performance Predicts Early Continence After Robot-Assisted Radical Prostatectomy.

“This study has provided the first evidence that surgeon performance in RARP is associated with early return to continence. Assessment of technical performance by a trained analyst using a valid metric of surgeon performance may be an independent predictor of patient outcomes in RARP.”

Impact of a simulation training curriculum on technical and nontechnical skills in colonoscopy: a randomized trial.

“These results support integration of endoscopy simulation into a structured curriculum incorporating instructional feedback and complementary didactic knowledge as a means to augment technical, cognitive, and integrative skills acquisition, as compared with SRL on virtual reality simulators.”

Evaluation of Deep Learning Models for Identifying Surgical Actions and Measuring Performance.

“The proposed models and the accompanying results illustrate that deep machine learning can identify associations in surgical video clips. These are the first steps to creating a feedback mechanism for surgeons that would allow them to learn from their experiences and refine their skills.”

Relationship between intraoperative non-technical performance and technical events in bariatric surgery.

“This study demonstrated that technical and non-technical performances are related, on both an individual and a team level. Valuable data can be obtained around intraoperative errors, events and rectifications.”

Acute mental stress and surgical performance.

“There is an association between measures of acute mental stress and worse technical surgical performance.”

Non-Technical Skills for Surgeons (NOTSS): Critical appraisal of its measurement properties.

“NOTSS is particularly suited for investigators who aim to provide ratings for surgeons and trainees with different levels of nontechnical skills and to track changes in the level of performance over a period of time, especially when educational interventions are involved. Thus, there is evidence for NOTSS to serve as a discriminative or evaluative instrument.”

Nontechnical Skill Assessment of the Collective Surgical Team Using the Non-Technical Skills for Surgeons (NOTSS) System.

“Surgical operations are performed by a coordinated team of surgeons and surgical trainees, and consequently, assessment of nontechnical skills may benefit from holistic assessment of the collective surgical team. We found that the NOTSS system, although developed for assessment of individual surgeons, was a useful tool for observing and rating the surgical team.”

Understanding and Assessing Nontechnical Skills in Robotic Urological Surgery: A Systematic Review and Synthesis of the Validity Evidence

"Nontechnical skill assessment tools specific to the robotic surgical ecosystem must be developed and studied across various contexts of surgical training. […] Nontechnical performance should be assessed in conjunction with technical ability as a comprehensive measure of robotic skill acquisition, as they may be independent contributors to end outcomes.”

Automated Methods of Technical Skill Assessment in Surgery: A Systematic Review

"In the era of Competence by Design medical education, we believe that automated methods represent the next step toward objective analysis of surgical trainees. Applications of these tools range from low-stakes assessments for trainee learning, to summative assessment at the surgical licensing examination level and beyond.”

Individualized deliberate practice on a virtual reality simulator improves technical performance of surgical novices in the operating room: a randomized controlled trial.

“A curriculum of deliberate individualized practice on a VR simulator improves technical performance in the OR. This has implications to greatly improve the feasibility of implementing simulation-based curricula in residency training programs”

Validation of a structured training and assessment curriculum for technical skill acquisition in minimally invasive surgery: a randomized controlled trial.

“Residents who participated in the curriculum surmounted the early portion of the learning curve during ex vivo training and outperformed conventionally trained residents in both the OR and during a simulated crisis scenario.”

Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology.

“[…] The long-term objective of this project is to create a validated evaluation tool that will be used in residency training programs, as well as for credentialing practicing surgeons.”

Non-technical skills and device-related interruptions in minimally invasive surgery.

“Communication and teamwork are intimately linked to device related interruptions. Communication failures occur often in the OR and are known to contribute to adverse event.”

Setting Performance Standards for Technical and Nontechnical Competence in General Surgery.

“Credible and reliable technical and nontechnical performance standards can be set in General Surgery for the laparoscopic cholecystectomy […] with a real need to implement summative assessments that will determine whether appropriate levels of performance have been reached”

Assessing the team's perception on human factors in the operating environment.

“[…] studies should focus on using objective data, such as derived by ORBB, to evaluate human factor behavior in the OR, and to define what type of human factors are most relevant and valuable to surgical safety, and to incorporate in team-based training.”

Quantifying the "Assistant Effect" in Robotic-Assisted Radical Prostatectomy (RARP): Measures of Technical Performance.

“Technical ability of bedside assistants is important to maximize surgeon performance, a finding that could potentially translate into better outcomes and fewer complications for patients.”

Quantifying recall bias in surgical safety: a need for a modern approach to morbidity and mortality reviews.

“Our data conclusively show that the surgeon and the fellow were unable to remember pertinent details from operations as few as 7 days earlier, despite reporting immediately postoperatively being mentally prepared and being allowed to adequately concentrate during the operation.”

Debriefing decreases mental workload in surgical crisis: A randomized controlled trial.

“Debriefing that includes a systematic approach to crisis after simulated, postoperative crisis situations may decrease mental workload. This spare mental capacity may help improve trainees' clinical decision-making, reduce human errors, and/or direct attention toward acquiring technical/operative skills.”

Characterising 'near miss' events in complex laparoscopic surgery through video analysis.

“Error awareness is essential in daily surgical practice and surgical training. The current study highlights the benefits of detailed video analysis to create a database of common injury mechanisms and video clip repository that can be used in tailoring future training interventions.”

Exploring stakeholder perceptions around implementation of the Operating Room Black Box for patient safety research

“While most stakeholders were supportive of the OR Black Box,[…] It is critical to ensure all stakeholders have adequate and accurate information about the OR Black Box system and research goals, and that the OR Black Box is positioned as a patient safety initiative.”

From box ticking to the black box: the evolution of operating room safety.

“The future of patient safety will be driven by interventions that harness technology to recognize patterns of performance or error that lead to patient harm and provide feedback to surgical teams that can be used to enhance future performance.”

Development and Evaluation of a Novel Instrument to Measure Severity of Intraoperative Events Using Video Data.

“We developed and evaluated a novel instrument to measure severity of intraoperative events using video data from laparoscopic gastrointestinal operations. We underwent a robust development phase, which included reviews of the literature and video clips to generate a taxonomy of intraoperative events.”

Factors associated with surgeon's perception of distraction in the operating room.

“[…] The “sterile cockpit” concept utilized in aviation, in which non-essential communication is prohibited during critical phases of flight, is an example of an implementable intervention that may be applied to surgery to lessen the impact of perceived distraction”

Adverse Events in the Operating Room: Definitions, Prevalence, and Characteristics. A Systematic Review.

“There is a lack of general agreement on how intraoperative adverse events (iAE) should be measured and reported. Our findings suggest that direct observation method has the most potential to identify and characterize iAE in detail.”

Feasibility of expert and crowd-sourced review of intraoperative video for quality improvement […] during robotic radical cystectomy.

“Self-debriefing was cost-effective in crisis resource management simulation training when compared to instructor debriefing. This study provides evidence regarding cost-effectiveness that […] may help to optimize resource allocation in education.”

A cost-effective junior resident training and assessment simulator for orthopaedic surgical skills via fundamentals of orthopaedic surgery.

“The Fundamentals of Orthopaedic Surgery (FORS) simulator was developed to help increase the practice of relevant orthopaedic tasks by junior residents in a cost-effective manner and thereby allow universal access to all residents.”

Design and validation of a low-cost, high-fidelity model for robotic pyeloplasty simulation training.

“The authors developed a simple, low-cost pyeloplasty model that is highly acceptable to trainees and faculty and demonstrates preliminary content and construct validity.”

The OR Black Box Nursing Education Curriculum: Using Video Review to Optimize Patient Safety.

“[…] we have developed a video-based curriculum for perioperative nurses based on patient safety themes such as distractions and communication. Increased awareness of factors affecting patient care has the potential to inform best practices and therefore optimize patient safety.”

Implementation of the Operating Room Black Box Research Program at the Ottawa Hospital Through Patient, Clinical, and Organizational Engagement: Case Study.

“As long as technology such as the Operating Room Black Box continues to be implemented by and for frontline health care providers in partnership with patients, it is likely to result in innovative quality improvement...”

Introduction of a surgical Black Box system in a hybrid angiosuite: Challenges and opportunities.

“In laparoscopy, the Operating Room Black Box® (ORBB) […] may also enhance endovascular surgical practice; however, the use of a C-arm and X-rays pose important challenges...”

Are We Ready for Our Close-up?: Why and How We Must Embrace Video in the OR.

“Unfiltered surgical data would be a boon to those of us studying the OR. […] We will be able to systematically assess new techniques and technologies, identify best practices among numerous individual preferences, and provide evidence-based recommendations...”

Video Technologies for Recording Open Surgery: A Systematic Review.

“Unfiltered surgical data would be a boon to those of us studying the OR. […] We will be able to systematically assess new techniques and technologies, identify best practices among numerous individual preferences, and provide evidence-based recommendations...”

Explainable Artificial Intelligence for Safe Intraoperative Decision Support.

“Explainable artificial intelligence could be a powerful tool for intraoperative decision support, used in warning systems to help clinicians predict and avoid adverse events that may lead to complications.”

Evaluation of Deep Learning Models for Identifying Surgical Actions and Measuring Performance.

“Our proposed neural network models inferred temporal patterns from surgical instrument motions and associated them with surgical gestures, actions, and performance-related cues given validated rating scales.”

Evaluation of a Virtual Reality Percutaneous Nephrolithotomy (PCNL) Surgical Simulator.

“The participants indicated that Marion Surgical K181 [virtual reality surgical sumulator] is an effective tool for surgery rehearsal, hand-eye coordination training, and that the simulator provides benefits over traditional training methods.”

Pushing boundaries of video review in trauma: using comprehensive data to improve the safety of trauma care.

“Although there are limited data on such a technology in the trauma bay, the Operating Room (OR) Black Box (Surgical Safety Technologies) has proven successful in characterizing adverse events and identifying latent safety threats in the OR.”

Within-team debriefing versus instructor-led debriefing for simulation-based education: a randomized controlled trial.

“Within-team debriefing may be as effective as instructor-led team debriefing for team-based simulation, which could improve cost-effectiveness and flexibility of scheduling of team-based simulation sessions.”

Simulation-trained junior residents perform better than general surgeons on advanced laparoscopic cases.

“The present study demonstrates that first year residents with laboratory bench model simulation training may technically perform advanced laparoscopic procedures in the OR.”

Evaluation of Procedural Simulation as a Training and Assessment Tool in General Surgery-Simulating a Laparoscopic Appendectomy.

“There were significant differences for task time, movements, and errors when comparing novices and more experience surgeons training on a simulator module for laparoscopic appendectomy […].”

Simulation-Based Laparoscopic Surgery Crisis Resource Management Training-Predicting Technical and Nontechnical Skills.

“resident level of training and laparoscopic experience correlated with technical performance during a simulation-based laparoscopic IVC injury crisis management scenario”

Surgical Education, Simulation, and Simulators-Updating the Concept of Validity.

“As the impact of assessment scores on clinical outcomes becomes more evident across multiple surgical specialties, acquiring consequences evidence will become central when designing an assessment for trainee technical or non-technical performance”

Progressive learning in endoscopy simulation training improves clinical performance: a blinded randomized trial.

“We report the first application of a progressive learning strategy to teaching colonoscopy. To our knowledge, this is the first comparative-effectiveness study to evaluate 2 different simulation curricula, with outcomes measured within the clinical setting.”

Simulation-Based Training of Non-Technical Skills in Colonoscopy: Protocol for a Randomized Controlled Trial.

“Non-technical skills training can improve surgical performance in the simulated setting, lead to better attitudes towards patient safety, and promote more self-reflection among trainees”

Virtual Surgical Training During COVID-19: Operating Room Simulation Platforms Accessible From Home.

“With a dramatic reduction in operative exposure, and in some cases, with a mandate to remain at home, how will surgical trainees maintain their surgical skill and intraoperative knowledge?”

Simulation in surgical education.

“Technical skills learned on low-fidelity benchtop models, video-based trainers and virtual-reality models are transferable to the operating room.”

Virtual reality in surgical skills training.

“Virtual-reality simulation allows for a more flexible environment than can be created by low- and highfidelity bench-top models. Many of the current VR systems allow for practice at varying levels of difficulty and across a wide range of clinical scenarios, thus accommodating learners at many levels.”

Simulation-based summative assessments in surgery.

“The use of summative simulation-based assessments requires development of appropriate assessment models and tools, training of the faculty, and standard setting.”

Comprehensive simulation-enhanced training curriculum for an advanced minimally invasive procedure: a randomized controlled trial.

“This prospective single-blinded randomized controlled trial demonstrated that completion of a comprehensive simulation-enhanced training curriculum for an advanced minimally invasive procedure resulted in superior psychomotor, and nontechnical skills, compared with conventional surgery training.”