Multi-camera operating room activity analysis for workflow analysis

Wanxin Xu; Albert Huang

doi:10.1117/12.2611586

4 April 2022 Multi-camera operating room activity analysis for workflow analysis

Wanxin Xu, Albert Huang

Proceedings Volume 12037, Medical Imaging 2022: Imaging Informatics for Healthcare, Research, and Applications; 120370B (2022) https://doi.org/10.1117/12.2611586
Event: SPIE Medical Imaging, 2022, San Diego, California, United States

Abstract

Recognizing workflow in the operating room (OR) is essential for developing context-aware systems to provide rich information to enhance patient outcomes by providing performance feedback to the staff post operation or by managing OR scheduling and utilization in real-time. However, manual event reporting can be unreliable and time-consuming. Therefore, our work aims to develop an automatic OR activity analysis framework for recognizing OR workflow for efficiency understanding, analysis and optimization. Such tasks are challenging due to the OR staff wearing similar scrubs and face masks in this setting. The presence of large machinery and equipment also occlude much camera views. By utilizing multiple pre-calibrated cameras mounted in the OR, our system localizes, tracks and recognizes the activity of each staff in OR. First, multi-person localization is done in 2D by using a top-down neural network. Next, an efficient and novel 3D reconstruction by tracking scheme is employed by matching 2D poses across different camera views with back-projected 2D poses from previous frame. Lastly, actions, such as standing, walking, sitting, crouching and lying, are recognized via prior pose knowledge and a pose graph convolution network. We evaluated pose estimation accuracy using percentage of correct key points (PCK) metric and achieved 78.0% accuracy on MVOR dataset, which outperformed state-of-art performance (76.2%). Based on these poses, the actions recognized are 96% accurate. Given the results, our proposed framework for automatic OR activity analysis demonstrates its potential to facilitate further online or offline workflow analysis.

Conference Presentation

Citation Download Citation

Wanxin Xu and Albert Huang "Multi-camera operating room activity analysis for workflow analysis", Proc. SPIE 12037, Medical Imaging 2022: Imaging Informatics for Healthcare, Research, and Applications, 120370B (4 April 2022); https://doi.org/10.1117/12.2611586

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available