Method and system for extracting an actual surgical duration from a total operating room (OR) time of a surgical procedure

Abstract

Embodiments described herein provide various examples of a system for extracting an actual procedure duration composed of actual surgical tool-tissue interactions from an overall procedure duration of a surgical procedure on a patient. In one aspect, the system is configured to obtain the actual procedure duration by: obtaining an overall procedure duration of the surgical procedure; receiving a set of operating room (OR) data from a set of OR data sources collected during the surgical procedure, wherein the set of OR data includes an endoscope video captured during the surgical procedure; analyzing the set of OR data to detect a set of non-surgical events during the surgical procedure that do not involve surgical tool-tissue interactions; extracting a set of durations corresponding to the set of non-surgical events; and determining the actual procedure duration by subtracting the set of extracted durations from the overall procedure duration.

What is claimed is: 
     
       1. A computer-implemented method for determining a total tool-tissue interaction duration of a surgical procedure on a patient, the method comprising the following operations performed by a digital processor:
 obtaining an overall procedure duration of the surgical procedure performed by a surgeon on the patient; 
 receiving operating room (OR) data collected during the surgical procedure, wherein the OR data includes an endoscope video captured by an endoscope during the overall procedure duration of the surgical procedure; 
 analyzing, by a machine-learning, ML, model the OR data for the overall procedure duration of the surgical procedure, including detecting all tool-tissue interaction events during the overall procedure duration, as a result of which the ML model identifies a plurality of surgical timeout events for the surgical procedure and a plurality of out of body events in which the endoscope is taken out of the body of the patient and inserted back into the body, wherein the ML model identifies each of the plurality of surgical timeout events whenever a surgical tool in the endoscope video has stopped moving for more than a predetermined time period due to the surgeon pausing a surgical task being performed on the patient and there is no surgical tool-tissue interaction; 
 extracting a plurality of durations of the plurality of surgical timeout events and a plurality of durations of the plurality of out of body events, respectively; 
 computing the total tool-tissue interaction duration based on subtracting the plurality of durations of the plurality of surgical timeout events including the plurality of durations of the plurality of out body events, from the overall procedure duration; and 
 for each of the plurality of out of body events, blurring a segment of the endoscope video that corresponds to an out of body event to anonymize the segment. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the OR data further comprises:
 audio recorded inside the OR during the surgical procedure; and 
 wherein the digital processor analyzes for the overall procedure duration the endoscope video and the recorded audio to thereby detect all intervals in which the endoscope video becomes static and the recorded audio contains a discussion between the surgeon and another person. 
 
     
     
       3. The computer-implemented method of  claim 1 , wherein the OR data further comprises audio recorded inside the OR during the surgical procedure, a video captured by a wall camera or a ceiling camera inside the OR during the surgical procedure and sensor data collected inside the OR during the surgical procedure, the sensor data comprises one or more of:
 pressure sensor data collected from surgical tools involved in the surgical procedure; 
 pressure sensor data collected from a surgical platform inside the OR; and 
 pressure sensor data collected from a doorway of the OR, and wherein 
 the digital processor analyzes for the overall procedure duration i) the endoscope video, ii) the recorded audio, iii) the video captured by the wall camera or the ceiling camera, and iv) the sensor data, to detect the plurality of surgical timeout events. 
 
     
     
       4. The computer-implemented method of  claim 1 , wherein extracting the plurality of durations of the surgical timeout events comprises for each duration and its associated surgical timeout event:
 extracting an initial time being when movement of the surgical tool is determined by the ML model to have stopped; and 
 extracting an end time being when movement the surgical tool is determined by the ML model to have resumed. 
 
     
     
       5. The computer-implemented method of  claim 4 , further comprising collaborating the initial time and the end time of the associated surgical timeout event with pressure sensor data collected from a pressure sensor located at a tip of the surgical tool. 
     
     
       6. The computer-implemented method of  claim 5 , wherein
 collaborating the initial time and the end time with the pressure sensor data comprises: collaborating the initial time with a first time when the pressure sensor data decreases to a first threshold; and 
 collaborating the end time with a second time when the pressure sensor data increases to a second threshold greater than the first threshold. 
 
     
     
       7. The computer-implemented method of  claim 4 , wherein the surgical timeout event occurs:
 when the surgeon stops interacting with the patient and starts a discussion with another surgeon, a surgical support team, or a resident surgeon; 
 when the surgeon pauses to make a decision on how to proceed with the surgical procedure based on an on-screen event or a surgical complication; or 
 when the surgeon pauses to wait for a collaborating surgeon to come into the OR. 
 
     
     
       8. The computer-implemented method of  claim 1 , wherein the ML model is to:
 identify the beginning of the OOB event based on a first sequence of video images in the endoscope video; and 
 identify the end of the OOB event based on a second sequence of video images in the endoscope video. 
 
     
     
       9. The computer-implemented method of  claim 1 , wherein the OOB event coincides with:
 cleaning a lens of the endoscope which blocks an endoscopic view; 
 changing the lens from one scope size to another scope size; or 
 switching the surgical procedure from a robotic surgical system to a laparoscopic surgical system. 
 
     
     
       10. The computer-implemented method of  claim 1 , further comprising:
 providing part of the OR data as input to another machine-learning model in response to which the other ML model detects a non-surgical event by
 identifying a pre-surgery patient preparation time prior to the surgical procedure; and 
 identifying a post-surgery patient assistant time after completion of the surgical procedure, 
 wherein determining the total tool-tissue interaction duration is further based on subtracting the pre-surgery patient preparation time and the post-surgery patient assistant time from the overall procedure duration. 
 
 
     
     
       11. The computer-implemented method of  claim 1 , wherein obtaining the overall procedure duration of the surgical procedure includes determining a time when the patient is being wheeled into the OR and a time when the patient is being wheeled out of the OR. 
     
     
       12. A system for determining a total tool-tissue interaction duration of a surgical procedure on a patient, the system comprising:
 a processor; and 
 a memory coupled to the processor, wherein the memory stores instructions that, when executed by the processor, cause the system to:
 obtain an overall procedure duration of the surgical procedure performed by a surgeon on the patient; 
 receive operating room (OR) data collected during the surgical procedure, wherein the OR data includes an endoscope video captured by an endoscope during the overall procedure duration of the surgical procedure; 
 analyzing, by a machine-learning model the OR data for the overall procedure duration of the surgical procedure, including detecting all tool-tissue interaction events during the overall procedure duration, as a result of which the ML model identifies a plurality of surgical timeout events for the surgical procedure and one or more out of body events in which the endoscope is taken out of the body of the patient and inserted back into the body, wherein the ML model identifies each of the plurality of surgical timeout events whenever a surgical tool in the endoscope video has stopped moving for more than a predetermined time period due to the surgeon pausing a surgical task being performed on the patient and there is no surgical tool-tissue interaction; 
 extract a plurality of durations of the plurality of surgical timeout events, respectively, and durations of the one or more out of body events, respectively; 
 compute the total tool-tissue interaction duration based on subtracting the plurality of durations of the plurality of surgical timeout events including the durations of the one or more out of body events, from the overall procedure duration; and 
 for each of the one or more out of body events, blurring a segment of the endoscope video that corresponds to an out of body event to anonymize the segment. 
 
 
     
     
       13. The system of  claim 12 , wherein the OR data further comprises:
 audio recorded inside the OR during the surgical procedure; and 
 wherein the processor analyzes for the overall procedure duration the endoscope video and the recorded audio to thereby detect all intervals in which the endoscope video becomes static and the recorded audio contains a discussion between the surgeon and another person. 
 
     
     
       14. The system of  claim 12 , wherein the OR data further comprises audio recorded inside the OR during the surgical procedure, a video captured by a wall camera or a ceiling camera inside the OR during the surgical procedure and sensor data collected inside the OR during the surgical procedure, the sensor data comprises one or more of:
 pressure sensor data collected from surgical tools involved in the surgical procedure; 
 pressure sensor data collected from a surgical platform inside the OR; and 
 pressure sensor data collected from a doorway of the OR and wherein 
 the processor analyzes for the overall procedure duration i) the endoscope video, ii) the recorded audio, iii) the video captured by the wall camera or the ceiling camera, and iv) the sensor data, to detect the plurality of surgical timeout events. 
 
     
     
       15. The system of  claim 12  wherein the instructions cause the system to extract the plurality of durations of the surgical timeout events by, for each duration and its associated surgical timeout event:
 extracting an initial time being when movement of the surgical tool is determined by the ML model to have stopped; and 
 extracting an end time being when movement the surgical tool is determined by the ML model to have resumed. 
 
     
     
       16. The system of  claim 15  wherein the instructions cause the system to collaborate the initial time and the end time of the associated surgical timeout event with pressure sensor data collected from a pressure sensor located at a tip of the surgical tool. 
     
     
       17. The system of  claim 15  wherein the surgical timeout event occurs:
 when the surgeon stops interacting with the patient and starts a discussion with another surgeon, a surgical support team, or a resident surgeon; 
 when the surgeon pauses to make a decision on how to proceed with the surgical procedure based on an on-screen event or a surgical complication; or 
 when the surgeon pauses to wait for a collaborating surgeon to come into the OR.