Surgical training apparatus, methods and systems

Abstract

Surgical training apparatus, methods and systems which allow surgical trainees to practice surgical skills on anatomical models in a realistic manner with an augmented reality headset and delivery of targeted surgical coursework curriculum correlated to the actions of the trainee as sensed by sensors in or adjacent the model to help the trainee develop proper surgical technique and decision making.

What is claimed is: 
     
         1 . A surgical training system, comprising:
 a) a model of an anatomical part of a human or animal;   b) one or more sensors attached to said model, said one or more sensors operable to emit a signal in response to receiving an activation input;   c) an augmented reality headset having one or more electronic and/or optical input and output channels adapted to receive electronic signals from said one or more sensors;   d) a computer processor operable to receive signals from said augmented reality headset and/or said one or more sensors;   e) a computer database having surgical training curriculum having one or more individual surgical subject matter components stored therein; and   f) a software program running on said computer processor and connected to said database, said software program operable to correlate said one or more sensor signals to said one or more individual surgical subject matter components, said software program being further operable to provide as an output the individual surgical subject matter component which is correlated to a received signal.   
     
     
         2 . The surgical training system of  claim 1 , wherein a machine learning model receives visual and/or electronic signal cues from one or more sensors attached to the model or the augmented reality headset. 
     
     
         3 . The surgical training system of  claim 2 , wherein the software program tracks the state of progress of steps of a surgical procedure from beginning to end. 
     
     
         4 . The surgical training system of  claim 3 , wherein each step of the surgical procedure has one or more machine learning models having two or more machine learning classes which determine transition to a next step of the surgical procedure. 
     
     
         5 . The surgical training system of  claim 4 , wherein the one or more machine learning models predict a class which corresponds to a correct action for a particular step of the surgical procedure, and an affirmative instruction is displayed to the student through a projected display waveguide with see through optics of the augmented reality headset for the particular step. 
     
     
         6 . The surgical training system of  claim 5 , wherein the one or more machine learning models predict a class which corresponds to an incorrect action for a particular step of the surgical procedure, and a corrective instruction is displayed to the student through the projected display waveguide with see through optics of the augmented reality headset for the particular step. 
     
     
         7 . The surgical training system of  claim 6 , wherein at the end of the surgical procedure, the software program is operable to tally all occurrences of correct actions and incorrect actions in order to produce a quantitative score which tracks the state of progress of each step of the surgical procedure. 
     
     
         8 . The surgical training system of  claim 7 , wherein at the end of the surgical procedure, a recording of the surgical procedure is saved so that it may be reviewed by the student and/or instructor for instructional or proficiency scoring purposes. 
     
     
         9 . A procedure training system comprising:
 (a) a physical model associated with performance of one or more procedures by a trainee, each of the one or more procedures having a finite number of defined steps, each step having one or more cues associated therewith;   (b) one or more cue receivers operable to receive one or more cues during performance of the one or more procedures associated with the physical model, and generate a signal in response to receiving the one or more cues;   (c) a computer processor operable to receive signals from said one or more cue receivers;   (d) a computer database having training curriculum having one or more individual subject matter components stored therein for each of the one or more procedures; and   (e) a software program running on said computer processor and connected to said database, said software program operable to correlate said received signals to said one or more individual subject matter components for a particular procedure, and provide as an output the individual subject matter component which is correlated to a received signal,   wherein the one or more cue receivers includes an augmented reality (AR) headset having one or more electronic and/or optical input and output channels adapted to send and receive said signals corresponding to said received cues.   
     
     
         10 . The procedure training system of  claim 9 , wherein the one or more cues include any one or combination of:
 model cues for detecting discrete elements or conditions associated with the physical model itself,   identification and/or motion (ID-motion) cues for detecting physical presence or motions by the trainee or an instrument used by the trainee, and/or   still picture/video cues including image capture and video feeds.   
     
     
         11 . The procedure training system of  claim 10 , wherein the one or more cue receivers further include any one or combination of:
 one or more image scanners,   one or more cameras operable to capture images and/or videos, and   one or more sensors in, on, near, or associated with the physical model.   
     
     
         12 . The procedure training system of  claim 11 , wherein the one or more cue receivers generate said signals based on any one or combination of a change in the physical model, a movement of the trainee, use of an instrument by the trainee, and/or a particular field of view during the course of the procedure. 
     
     
         13 . The procedure training system of  claim 10 , wherein the software program is programmed with one or more machine learning (ML) models operable to identify, process, and classify the received signals for each step of the one or more procedures,
 wherein the one or more ML models have two or more classes which determine a transition to a next step from a current step of a particular procedure, and   wherein the one or more ML models enable the software program to determine and communicate through the AR headset or a computer if actions performed by the trainee for each step of the particular procedure are correct or incorrect.   
     
     
         14 . The procedure training system of  claim 13 , wherein the one or more ML models are operable to detect specific performance of the trainee during the course of the procedure associated with the physical model based on the received signals associated with one or more of the model cues, the ID-motion cues and/or the still picture/video cues, and the software program is operable to output corresponding training curriculum, alerts, and/or related instructions to the trainee based on the specific performance of the trainee. 
     
     
         15 . The procedure training system of  claim 14 , wherein the software program is operable to interpret the received signals as a specific action or reference point of the physical model within the context of each step of the particular procedure using the one or more ML models, and cause a state change during the performance of the procedure by the trainee that will advance the procedure from the current step to the next step. 
     
     
         16 . The procedure training system of  claim 15 , wherein the software program is programmed to deliver tutorials and guidance to the trainee that correspond to ongoing progress of the steps of the procedure. 
     
     
         17 . The procedure training system of  claim 15 , wherein the software program is programmed with an ordered sequence of actions on the physical model which are indicative of a successful procedure, and wherein the model cues and/or the ID-motion cues and/or the still picture/video cues are correlated to the programmed ordered sequence of actions for the particular procedure. 
     
     
         18 . The procedure training system of  claim 17 , wherein the software program is operable to identify whether the trainee performs any actions on the physical model that are not in agreement with an expected protocol or performance standard as identified in the software program based on the one or more ML models, and inform the trainee of a detected digression from the expected protocol or performance standard. 
     
     
         19 . The procedure training system of  claim 18 , wherein the software program is operable to deliver corresponding training curriculum, an alert, and/or corrective instructions to the trainee at the time of the detected digression from the expected protocol or performance standard during the performance of the procedure and/or at the conclusion of the particular procedure. 
     
     
         20 . The procedure training system of  claim 15 , wherein the software program is operable to analyze the received signals from the one or more cue receivers using the one or more ML models corresponding to the current step of the particular procedure, and determine whether the received signals indicate that the trainee performed the current step of the procedure properly. 
     
     
         21 . The procedure training system of  claim 20 , wherein the software program is operable to:
 output corresponding training curriculum and/or affirmative instructions associated with the next step in the particular procedure in response to the received signals indicating that the trainee performed the current step of the procedure properly, or   output corresponding training curriculum and/or corrective instructions associated with the current step in the particular procedure in response to the received signals indicating that the trainee did not perform the current step properly.   
     
     
         22 . The procedure training system of  claim 13 , wherein the software program is operable to generate and output a performance score or figure of merit for individual steps taken by the trainee and/or for the entire procedure based on the number of correct actions and incorrect actions detected using the one or more ML models. 
     
     
         23 . The procedure training system of  claim 13 , wherein the software program is operable to log, timestamp, and store the detected signals in the database based on recognized classifications using the one or more ML models, and provide the ability to playback recordings of the procedures for training debrief purposes after completion of the particular procedure. 
     
     
         24 . The procedure training system of  claim 10 , wherein the software program is programmed with direct visual detection algorithms including machine learning, deep learning, and/or reinforcement learning to develop cue detection functions. 
     
     
         25 . The procedure training system of  claim 10 , wherein training of the software program is an offline process performed by taking the cues of interest in identifying quality of performance of each step of the procedure by using machine learning software for image process training. 
     
     
         26 . The procedure training system of  claim 10 , wherein the software program is trained for machine and deep learning using neural networks for detection of user technique during performance of the steps of the procedure,
 wherein a neural network classifies patterns of images and/or signals based on a learned features database defining two or more classes for each step of the procedure, and outputs a performance score or figure of merit for each step of the procedure based on the classified patterns.   
     
     
         27 . The procedure training system of  claim 10 , wherein the software program is operable to detect a current step of the particular procedure based on one or more detected model cues and/or ID-motion cues and/or still picture/video cues. 
     
     
         28 . The procedure training system of  claim 27 , wherein training of the software program and/or machine learning model is an offline process performed by further segregating recorded detected model cues and/or ID-motion cues and/or still picture/video cues for a specific step of the particular procedure into machine learning classes utilizing unsupervised learning, by utilizing machine learning models that can accurately predict classes with high confidence which were trained by supervised learning as a method of segregating recorded information into machine learning model classes. 
     
     
         29 . The procedure training system of  claim 28 , wherein machine learning models are created for each step of the procedure with a greater level of confidence than the machine learning models obtained by supervised learning, by utilizing unsupervised learning and utilizing class data obtained during the offline process.