Surgical teaching auxiliary system using virtual reality and method thereof

Abstract

Surgical teaching auxiliary system using virtual reality and method thereof are disclosed. In the Surgical teaching auxiliary system, virtual-reality surgical environment is created through three-dimensional reconstruction based on a two-dimensional surgical image, and an operation step, a time point of using instrument and a reference message are preset in advance; when a surgical training is performed in the virtual-reality surgical environment, a surgical operation behavior is continuously detected; when the surgical operation behavior abnormally interrupted or delayed is detected, an auxiliary support and guidance are triggered based on a progress of the surgical training, thereby achieving the technical effect of greatly improving the effectiveness of surgical learning in the virtual reality.

What is claimed is: 
     
         1 . A surgical teaching auxiliary system using virtual reality, comprising:
 a surgical image database, configured to store two-dimensional surgical images, wherein each of the two-dimensional surgical images corresponds to a surgical method name, an operation step, a time point of using an instrument, and a reference message; and   a host, linked to the surgical image database, wherein the host comprises:
 a non-transitory computer-readable storage medium, configured to store computer-readable program instructions; and 
 a hardware processor, electrically connected to the non-transitory computer-readable storage medium, and configured to execute the computer-readable program instructions to execute:
 a creating module, linked to the surgical image database, wherein before a surgical training is performed, the creating module is configured to receive the surgical method name, load one of the two-dimensional surgical images that corresponds to the received surgical method name, and create virtual-reality surgical environment through 3D reconstruction based on the loaded two-dimensional surgical image; 
 a training module, linked to the creating module, wherein after the virtual-reality surgical environment is created completely, the training module is configured to perform a surgical training, wherein when the surgical training is performed, the training module is configured to continuously detect a surgical operation behavior, and generate a time point message based on a progress of the surgical training when the surgical operation behavior is abnormally interrupted or delayed; and 
 a supporting module, linked to the training module, and configured to display the loaded two-dimensional surgical image based on the generated time point message, and synchronously display the operation step, the time point of using instrument, and the reference message, to provide an auxiliary support and guidance. 
 
   
     
     
         2 . The surgical teaching auxiliary system using virtual reality according to  claim 1 , wherein the training module inputs the detected surgical operation behavior into a machine learning model which is trained completely, to recognize whether the detected surgical operation behavior is abnormally interrupted or delayed, through the machine learning model. 
     
     
         3 . The surgical teaching auxiliary system using virtual reality according to  claim 2 , wherein the machine learning model which is trained completely is produced by continuously inputting the surgical operation behavior that has no abnormally interrupted or delayed condition into the machine learning model that has not been trained. 
     
     
         4 . The surgical teaching auxiliary system using virtual reality according to  claim 1 , wherein the three-dimensional reconstruction is performed through at least one of optical measure, geometry, deep learning algorithm, to transform the loaded two-dimensional surgical image into 3D information of virtual reality. 
     
     
         5 . The surgical teaching auxiliary system using virtual reality according to  claim 1 , wherein the surgical operation behavior is detected by at least one of a motion sensor, a wrist-worn device, and a hand motion input device. 
     
     
         6 . A surgical teaching auxiliary method using virtual reality, wherein the surgical teaching auxiliary method is executed by a host linked to a surgical image database and comprises:
 storing two-dimensional surgical images in the surgical image database, wherein each of the two-dimensional surgical images corresponds to a surgical method name, an operation step, a time point of using instrument, and a reference message;   before a surgical training is performed, receiving a surgical method name to load one of the two-dimensional surgical image that corresponds to the surgical method name, and creating virtual-reality surgical environment through 3D reconstruction based on the loaded two-dimensional surgical image, by the host;   after the virtual-reality surgical environment is created completely, permitting to perform the surgical training, continuously detecting a surgical operation behavior when the surgical training is performed, generating a time point message based on a progress of the surgical training when the surgical operation behavior is interrupted or delayed abnormally, by the host; and   displaying the loaded two-dimensional surgical image, synchronously displaying the operation step, the time point of using instrument, and the reference message for auxiliary support and guidance based on the generated time point message, by the host.   
     
     
         7 . The surgical teaching auxiliary method using virtual reality according to  claim 6 , further comprising:
 inputting the detected surgical operation behavior into a machine learning model which is trained completely, to recognize whether the detected surgical operation behavior is abnormally interrupted or delayed, through the machine learning model, by the training module.   
     
     
         8 . The surgical teaching auxiliary method using virtual reality according to  claim 7 , wherein the machine learning model which is trained completely is produced by continuously inputting the surgical operation behavior that has no abnormally interrupted or delayed condition into the machine learning model that has not been trained. 
     
     
         9 . The surgical teaching auxiliary method using virtual reality according to  claim 6 , wherein the three-dimensional reconstruction is performed through at least one of optical measure, geometry, deep learning algorithm, to transform the loaded two-dimensional surgical image into 3D information of virtual reality. 
     
     
         10 . The surgical teaching auxiliary method using virtual reality according to  claim 6 , wherein the surgical operation behavior is detected by at least one of a motion sensor, a wrist-worn device, and a hand motion input device.