Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published September 2022 | public
Journal Article

Use of surgical video–based automated performance metrics to predict blood loss and success of simulated vascular injury control in neurosurgery: a pilot study

Abstract

Objective: Experts can assess surgeon skill using surgical video, but a limited number of expert surgeons are available. Automated performance metrics (APMs) are a promising alternative but have not been created from operative videos in neurosurgery to date. The authors aimed to evaluate whether video-based APMs can predict task success and blood loss during endonasal endoscopic surgery in a validated cadaveric simulator of vascular injury of the internal carotid artery. Methods: Videos of cadaveric simulation trials by 73 neurosurgeons and otorhinolaryngologists were analyzed and manually annotated with bounding boxes to identify the surgical instruments in the frame. APMs in five domains were defined—instrument usage, time-to-phase, instrument disappearance, instrument movement, and instrument interactions—on the basis of expert analysis and task-specific surgical progressions. Bounding-box data of instrument position were then used to generate APMs for each trial. Multivariate linear regression was used to test for the associations between APMs and blood loss and task success (hemorrhage control in less than 5 minutes). The APMs of 93 successful trials were compared with the APMs of 49 unsuccessful trials. Results: In total, 29,151 frames of surgical video were annotated. Successful simulation trials had superior APMs in each domain, including proportionately more time spent with the key instruments in view (p 2 value of 0.87 (p < 0.001). Conclusions: Video-based APMs were superior predictors of simulation trial success and blood loss than surgeon characteristics such as case volume and attending status. Surgeon educators can use APMs to assess competency, quantify performance, and provide actionable, structured feedback in order to improve patient outcomes. Validation of APMs provides a benchmark for further development of fully automated video assessment pipelines that utilize machine learning and computer vision.

Additional Information

© 2021 American Association of Neurological Surgeons. Online Publication Date: 31 Dec 2021. Author Contributions Conception and design: Pangal, Kugener, Zhu, Zada, Donoho. Acquisition of data: Pangal, Kugener, Lechtholz-Zey, Collet, Lasky, Sundaram, Chan, Zada, Donoho. Analysis and interpretation of data: Pangal, Kugener, Lechtholz-Zey, Collet, Lasky, Sundaram, Zhu, Roshannai, Chan, Donoho. Drafting the article: Pangal, Kugener, Donoho. Critically revising the article: Pangal, Kugener, Cardinal, Roshannai, Chan, Sinha, Hung, Anandkumar, Zada, Donoho. Reviewed submitted version of manuscript: Pangal, Kugener, Cardinal, Sinha, Hung, Anandkumar, Zada, Donoho. Approved the final version of the manuscript on behalf of all authors: Pangal. Statistical analysis: Pangal, Kugener, Cardinal, Donoho. Administrative/technical/material support: Zada, Donoho. Study supervision: Zada, Donoho. Disclosures. Dr. Hung is a consultant for Johnson and Johnson, Mimic Technologies, and Quantgene.

Additional details

Created:
August 20, 2023
Modified:
October 23, 2023