QUALITY IMPROVEMENT

Using OR Black Box Technology to Determine Quality Improvement Outcomes

For In-situ Timeout and Debrief Simulation

For In-situ Timeout and Debrief Simulation

For In-situ Timeout and Debrief Simulation

Published on

July 11, 2024

Annals of Surgery

Campbell, Krystle K. DHA; Abreu, Andres A. MD; Zeh, Herbert J. MD; Daniel, William C. MD, MBA; Palter, Vanessa N. MD, PhD; Bishop, Samantha J. BSN, RN; Sims, Suzanne RN; Odeh, Jaffer M. MD; Evans, Kim MBA; Dandekar, Priya MBA; Scott, Daniel J. MD
Campbell, Krystle K. DHA; Abreu, Andres A. MD; Zeh, Herbert J. MD; Daniel, William C. MD, MBA; Palter, Vanessa N. MD, PhD; Bishop, Samantha J. BSN, RN; Sims, Suzanne RN; Odeh, Jaffer M. MD; Evans, Kim MBA; Dandekar, Priya MBA; Scott, Daniel J. MD
Campbell, Krystle K. DHA; Abreu, Andres A. MD; Zeh, Herbert J. MD; Daniel, William C. MD, MBA; Palter, Vanessa N. MD, PhD; Bishop, Samantha J. BSN, RN; Sims, Suzanne RN; Odeh, Jaffer M. MD; Evans, Kim MBA; Dandekar, Priya MBA; Scott, Daniel J. MD

Overview

This study examined the effectiveness of implementing in-situ simulation training to improve surgical safety checklist performance, utilizing the OR Black Box® for assessment. The research employed a prospective convergent multi-methods approach over a 6-month period, including pre-intervention, during-intervention, and post-intervention phases.

The results demonstrated significant improvements in debrief-related metrics for surgical teams that participated in the simulation training compared to those who did not. Specifically, the simulation group showed better debrief quality, compliance, and engagement scores. However, there were no significant differences in timeout scores between the groups. The study also revealed two primary thematic categories from the simulation discussions: "culture of safety" and "policy." The researchers concluded that the simulation-based quality improvement intervention created a psychologically safe training environment for operating room teams, and the use of OR Black Box proved valuable in analyzing outcomes and demonstrating the effectiveness of the training for participating surgeons.

Results

Scores were significantly better for the Sim-group compared with the No-sim group for debrief quality (84% vs. 79% P<0.001, during-intervention), compliance (73% vs. 66%, P<0.001, post-intervention), and engagement (80% vs. 73%, P=0.012, during-intervention). There were no between-group differences for Timeout scores.