System, Method and Computer Readable Medium for Determining Characteristics Of Surgical Related Items and Procedure Related Items Present for Use in the Perioperative Period

Abstract

A system and method to determine characteristics of surgical related items and procedure related items present for use in the perioperative period. The system and method may apply computer vision for determining status and tracking of the surgical related items and procedure related items, as well as related clinical, logistical and operational events in the perioperative period. The system and method provides for an intuitive, automated, and transparent tracking of sterile surgical items (SSI) such as single-use, sterile surgical supplies (SUSSS) and sterile surgical instruments, and quantification of SSI waste. In doing so, the system and method empowers administrators to reduce costs and surgeons to demonstrate usage of important equipment. The system and method removes the guesswork from monitoring and minimizing SSI waste and puts the emphasis on necessity and efficiency.

What is claimed is: 
     
         1 . A system configured for determining one or more characteristics of surgical related items and/or procedure related items present at preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings, comprising:
 one or more computer processors;   a memory configured to store instructions that are executable by said one or more computer processors, wherein said one or more computer processors are configured to execute the instructions to:
 receive settings image data corresponding with the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings; 
 run a trained computer vision model on the received settings image data to identify and label the surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings; 
 interpret the surgical related items and/or procedure related items through tracking and analyzing said identified and labeled surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings, to determine said one or more characteristics of the surgical related items and/or procedure related items; and 
 transmit said one or more determined characteristics to a secondary source. 
   
     
     
         2 . The system of  claim 1 , wherein said one or more computer processors are configured to execute the instructions to:
 retrain said trained computer vision model using said received settings image data from the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings.   
     
     
         3 . The system of  claim 2 , wherein said trained computer vision model is generated on preliminary image data using a machine learning algorithm. 
     
     
         4 . The system of  claim 3 , wherein: said training of said computer vision model, may be performed with one or more of the following configurations:
 i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         5 . The system of  claim 2 , wherein: said retraining of said computer vision model, may be performed with one or more of the following configurations:
 i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         6 . The system of  claim 1 , wherein said trained computer vision model is generated on preliminary image data using a machine learning algorithm. 
     
     
         7 . The system of  claim 6 , wherein: said training of said computer vision model, may be performed with one or more of the following configurations:
 i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         8 . The system of  claim 1 , wherein one or more of the following instructions:
 a) said receiving of said settings image data,   b) said running of said trained computer vision model, and   c) said interpreting of the surgical related items and/or procedure related items,   may be performed with one or more of the following configurations:   i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         9 . The system of  claim 1 , wherein said tracking and analyzing comprises one or more of the following:
 object identification for tracking and analyzing;   motion sensing for tracking and analyzing;   depth and distance assessment for tracking and analyzing; and   infrared sensing for tracking and analyzing.   
     
     
         10 . The system of  claim 1 , wherein said tracking and analyzing comprises specified multiple tracking and analyzing models. 
     
     
         11 . The system of  claim 1 , wherein said one or more computer processors are configured to execute the instructions for said tracking and analyzing at one or more of the following:
 one or more databases;   cloud infrastructure; and   edge-computing.   
     
     
         12 . The system of  claim 1 , wherein said secondary source includes one or more of any one of the following:
 local memory;   remote memory; or   display or graphical user interface.   
     
     
         13 . The system of  claim 1 , wherein the machine learning algorithm includes an artificial neural network (ANN) or deep learning algorithm. 
     
     
         14 . The system of  claim 13 , wherein said artificial neural network (ANN) includes:
 convolutional neural network (CNN); and/or   recurrent neural networks (RNN).   
     
     
         15 . The system of  claim 1 , wherein said determined one or more characteristics includes any combination of one or more of the following:
 identification of the one or more of the surgical related items and/or procedure related items;   usage or non-usage status of the one or more of the surgical related items and/or procedure related items;   opened or unopened status of the one or more of the surgical related items and/or procedure related items;   moved or non-moved status of the one or more of the surgical related items and/or procedure related items;   single-use or reusable status of the one or more of the surgical related items and/or procedure related items; or   association of clinical events, logistical events, or operational events.   
     
     
         16 . The system of  claim 1 , further comprising:
 one or more cameras configured to capture the image to provide said received image data.   
     
     
         17 . The system of  claim 1 , wherein said one or more computer processors are further configured to, based on said determined one or more characteristics, execute the instructions to:
 determine an actionable output to reduce unnecessary waste of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative phase, and/or postoperative settings;   determine an actionable output to reorganize the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to reduce supply, storage, sterilization and disposal costs associated with use of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to reduce garbage and unnecessary re-sterilization associated with use of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to streamline setup of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to improve efficiency of using the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to identify, rank, and/or recognize level of efficiency of surgeons or clinicians; and/or   determine an actionable output to improve the level of efficiency of using the surgical related items and/or procedure related items that are sterilized.   
     
     
         18 . The system of  claim 1 , wherein neither machine readable markings on the surgical related items and/or procedure related items nor communicable coupling between said system and the surgical related items and/or procedure related items are required by said system to provide said one or more determined characteristics. 
     
     
         19 . The system of  claim 1 , wherein said settings image data comprises information from the visible light spectrum and/or invisible light spectrum. 
     
     
         20 . The system of  claim 1 , wherein said settings image data comprises three dimensional renderings or representation of information of the surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings. 
     
     
         21 . A computer-implemented method for determining one or more characteristics of surgical related items and/or procedure related items present at preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings, comprising:
 receiving settings image data corresponding with the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings;   running a trained computer vision model on the received settings image data to identify and label the surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings;   interpreting the surgical related items and/or procedure related items through tracking and analyzing said identified and labeled surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings, to determine said one or more characteristics of the surgical related items and/or procedure related items; and   transmitting said one or more determined characteristics to a secondary source.   
     
     
         22 . The method of  claim 21 , further comprising:
 retraining said trained computer vision model using said received settings image data from the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings.   
     
     
         23 . The method of  claim 22 , wherein said trained computer vision model is generated on preliminary image data using a machine learning algorithm. 
     
     
         24 . The method of  claim 23 , wherein: said training of said computer vision model, may be performed with one or more of the following configurations:
 i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         25 . The method of  claim 22 , wherein: said retraining of said computer vision model, may be performed with one or more of the following configurations:
 i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         26 . The method of  claim 21 , wherein said trained computer vision model is generated on preliminary image data using a machine learning algorithm. 
     
     
         27 . The method of  claim 26 , wherein: said training of said computer vision model, may be performed with one or more of the following configurations:
 i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         28 . The method of  claim 21 , wherein one or more of the following actions:
 a) said receiving of said settings image data,   b) said running of said trained computer vision model, and   c) said interpreting of the surgical related items and/or procedure related items,   may be performed with one or more of the following actions:   i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         29 . The method of  claim 21 , wherein said tracking and analyzing comprises one or more of the following:
 object identification for tracking and analyzing;   motion sensing for tracking and analyzing;   depth and distance assessment for tracking and analyzing; and   infrared sensing for tracking and analyzing.   
     
     
         30 . The method of  claim 21 , wherein said tracking and analyzing comprises specified multiple tracking and analyzing models. 
     
     
         31 . The method of  claim 21 , wherein said tracking and analyzing may be performed with one or more of the following:
 one or more databases;   cloud infrastructure; and   edge-computing.   
     
     
         32 . The method of  claim 21 , wherein said secondary source includes one or more of any one of the following:
 local memory;   remote memory; or   display or graphical user interface.   
     
     
         33 . The method of  claim 21 , wherein the machine learning algorithm includes an artificial neural network (ANN) or deep learning algorithm. 
     
     
         34 . The method of  claim 33 , wherein said artificial neural network (ANN) includes:
 convolutional neural network (CNN); and/or   recurrent neural networks (RNN).   
     
     
         35 . The method of  claim 21 , wherein said determined one or more characteristics includes any combination of one or more of the following:
 identification of the one or more of the surgical related items and/or procedure related items;   usage or non-usage status of the one or more of the surgical related items and/or procedure related items;   opened or unopened status of the one or more of the surgical related items and/or procedure related items;   moved or non-moved status of the one or more of the surgical related items and/or procedure related items;   single-use or reusable status of the one or more of the surgical related items and/or procedure related items; or   association of clinical events, logistical events, or operational events.   
     
     
         36 . The method of  claim 21 , further comprising:
 one or more cameras configured to capture the image to provide said received image data.   
     
     
         37 . The method of  claim 21 , wherein based on said determined one or more characteristics, further comprising:
 determining an actionable output to reduce unnecessary waste of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative phase, and/or postoperative settings;   determining an actionable output to reorganize the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings or the simulated preoperative, intraoperative, and/or postoperative settings;   determining an actionable output to reduce supply, storage, sterilization and disposal costs associated with use of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determining an actionable output to reduce garbage and unnecessary re-sterilization associated with use of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determining an actionable output to streamline setup of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determining an actionable output to improve efficiency of using the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determining an actionable output to identify, rank, and/or recognize level of efficiency of surgeons or clinicians; and/or   determining an actionable output to improve the level of efficiency of using the surgical related items and/or procedure related items that are sterilized.   
     
     
         38 . The method of  claim 21 , wherein neither machine readable markings on the surgical related items and/or procedure related items nor communicable coupling between said surgical related items and/or procedure related items and a system associated with said method are required by said method to provide said one or more determined characteristics. 
     
     
         39 . The method of  claim 21 , wherein said settings image data comprises information from the visible light spectrum and/or invisible light spectrum. 
     
     
         40 . The method of  claim 21 , wherein said settings image data comprises three dimensional renderings or representation of information of the surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings. 
     
     
         41 . A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for determining one or more characteristics of surgical related items and/or procedure related items present at preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings, comprising:
 receiving settings image data corresponding with the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings;   running a trained computer vision model on the received settings image data to identify and label the surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings;   interpreting the surgical related items and/or procedure related items through tracking and analyzing said identified and labeled surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings, to determine said one or more characteristics of the surgical related items and/or procedure related items; and   transmitting said one or more determined characteristics to a secondary source.   
     
     
         42 . The non-transitory computer-readable medium of  claim 41 , further comprising:
 training said trained computer vision model using said received settings image data from the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings.   
     
     
         43 . The non-transitory computer-readable medium of  claim 42 , wherein said trained computer vision model is generated on preliminary image data using a machine learning algorithm. 
     
     
         44 . The non-transitory computer-readable medium of  claim 43 , wherein:
 said training of said computer vision model, may be performed with one or more of the following configurations:   i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         45 . The non-transitory computer-readable medium of  claim 42 , wherein:
 said retraining of said computer vision model, may be performed with one or more of the following configurations:   i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         46 . The non-transitory computer-readable medium of  claim 41 , wherein said trained computer vision model is generated on preliminary image data using a machine learning algorithm. 
     
     
         47 . The non-transitory computer-readable medium of  claim 46 , wherein:
 said training of said computer vision model, may be performed with one or more of the following configurations:   i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         48 . The non-transitory computer-readable medium of  claim 41 , wherein one or more of the following actions:
 a) said receiving of said settings image data,   b) said running of said trained computer vision model, and   c) said interpreting of the surgical related items and/or procedure related items,   may be performed with one or more of the following actions:   i) streaming to the cloud and in real-time,   ii) streaming to the cloud and in delayed time,   iii) aggregated and delayed,   iv) locally on an edge-computing node, and   v) locally and/or remotely on a network and/or server.   
     
     
         49 . The non-transitory computer-readable medium of  claim 41 , wherein said tracking and analyzing comprises one or more of the following:
 object identification for tracking and analyzing;   motion sensing for tracking and analyzing;   depth and distance assessment for tracking and analyzing; and   infrared sensing for tracking and analyzing.   
     
     
         50 . The non-transitory computer-readable medium of  claim 41 , wherein said tracking and analyzing comprises specified multiple tracking and analyzing models. 
     
     
         51 . The non-transitory computer-readable medium of  claim 41 , wherein said tracking and analyzing may be configured to be performed with one or more of the following:
 one or more databases;   cloud infrastructure; and   edge-computing.   
     
     
         52 . The non-transitory computer-readable medium of  claim 41 , wherein said secondary source includes one or more of any one of the following:
 local memory;   remote memory; or   display or graphical user interface.   
     
     
         53 . The non-transitory computer-readable medium of  claim 41 , wherein the machine learning algorithm includes an artificial neural network (ANN) or deep learning algorithm. 
     
     
         54 . The non-transitory computer-readable medium of  claim 53 , wherein said artificial neural network (ANN) includes:
 convolutional neural network (CNN); and/or   recurrent neural networks (RNN).   
     
     
         55 . The non-transitory computer-readable medium of  claim 41 , wherein said determined one or more characteristics includes any combination of one or more of the following:
 identification of the one or more of the surgical related items and/or procedure related items;   usage or non-usage status of the one or more of the surgical related items and/or procedure related items;   opened or unopened status of the one or more of the surgical related items and/or procedure related items;   moved or non-moved status of the one or more of the surgical related items and/or procedure related items;   single-use or reusable status of the one or more of the surgical related items and/or procedure related items; or   association of clinical events, logistical events, or operational events.   
     
     
         56 . The non-transitory computer-readable medium of  claim 41 , further comprising:
 one or more cameras configured to capture the image to provide said received image data.   
     
     
         57 . The non-transitory computer-readable medium of  claim 41 , wherein said one or more computer processors are further configured to, based on said determined one or more characteristics, execute the instructions to:
 determine an actionable output to reduce unnecessary waste of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative phase, and/or postoperative settings;   determine an actionable output to reorganize the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to reduce supply, storage, sterilization and disposal costs associated with use of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to reduce garbage and unnecessary re-sterilization associated with use of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to streamline setup of the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to improve efficiency of using the surgical related items for use in the preoperative, intraoperative, and/or postoperative settings and/or the simulated preoperative, intraoperative, and/or postoperative settings;   determine an actionable output to identify, rank, and/or recognize level of efficiency of surgeons or clinicians; and/or   determine an actionable output to improve the level of efficiency of using the surgical related items and/or procedure related items that are sterilized.   
     
     
         58 . The non-transitory computer-readable medium of  claim 41 , wherein neither machine readable markings on the surgical related items and/or procedure related items nor communicable coupling between said surgical related items and/or procedure related items and a system associated with said computer readable medium are required by said system to provide said one or more determined characteristics. 
     
     
         59 . The non-transitory computer-readable medium of  claim 41 , wherein said settings image data comprises information from the visible light spectrum and/or invisible light spectrum. 
     
     
         60 . The non-transitory computer-readable medium of  claim 41 , wherein said settings image data comprises three dimensional renderings or representation of information of the surgical related items and/or procedure related items in the preoperative, intraoperative, and/or postoperative settings and/or simulated preoperative, intraoperative, and/or postoperative settings.