Augmented surgical reality environment

Abstract

The present disclosure is directed to an augmented reality surgical system for viewing an augmented image of a region of interest during a surgical procedure. The system includes an image capture device that captures an image of the region of interest. A controller receives the image and applies at least one image processing filter to the image. The image processing filter includes a spatial decomposition filter that decomposes the image into spatial frequency bands. A temporal filter is applied to the spatial frequency bands to generate temporally filtered bands. An adder adds each band spatial frequency band to a corresponding temporally filtered band to generate augmented bands. A reconstruction filter generates an augmented image by collapsing the augmented bands. A display displays the augmented image to a user.

1 - 20 . (canceled) 
     
     
         21 . An augmented reality surgical system for viewing an augmented image of a region of interest, the system comprising:
 an image capture device configured to capture an image of the region of interest inside a patient during a surgical procedure;   a controller configured to receive the captured image, receive one or more desired frequency bands, and apply at least one image processing filter to the image to generate the augmented image, the image processing filter including:
 a spatial decomposition filter configured to decompose the image into a plurality of spatial frequency bands; 
 a temporal filter configured to extract the one or more desired spatial frequency bands from the plurality of spatial frequency bands; 
 an amplifier configured to amplify the one or more desired spatial frequency bands; 
 an adder configured to add each band in the amplified one or more desired spatial frequency bands to a corresponding band in the plurality of spatial frequency bands to generate a plurality of augmented bands; and 
 a reconstruction filter configured to generate an augmented image by collapsing the plurality of augmented bands; and 
   a display configured to display the augmented image thereon.   
     
     
         22 . The augmented reality surgical system of  claim 21 , wherein the one or more desired frequency bands are received via a user interface. 
     
     
         23 . The augmented reality surgical system of  claim 21 , wherein the one or more desired frequency bands are related to the region of interest. 
     
     
         24 . The augmented reality surgical system of  claim 21 , wherein the image capture device captures a video having a plurality of image frames and the controller applies the at least one image processing filter to each image frame of the plurality of image frames. 
     
     
         25 . The augmented reality surgical system of  claim 21 , wherein the temporal filter includes a bandpass filter. 
     
     
         26 . The augmented reality surgical system of  claim 21 , wherein the temporal filter isolates the one or more desired spatial frequency bands from the plurality of spatial frequency bands. 
     
     
         27 . The augmented reality surgical system of  claim 21 , wherein the image processing filter uses an edge detection algorithm configured to highlight one or more edges in the image,
 wherein the one or more highlighted edges is added to the augmented image.   
     
     
         28 . The augmented reality surgical system of  claim 21 , further comprising at least one hyper-spectral sensor configured to obtain a plurality of hyper-spectral images,
 wherein the image processing filter uses a hyper-spectral algorithm to combine the plurality of spectral images to generate a three dimensional hyper-spectral image cube that is added to the augmented image.   
     
     
         29 . The augmented reality surgical system of  claim 21 , further comprising an infrared camera configured to capture an infrared image,
 wherein the infrared image is added to the augmented image.   
     
     
         30 . A method for generating an augmented image of a region of interest, the method comprising:
 capturing an image of the region of interest inside a patient during a surgical procedure;   receiving one or more desired frequency bands;   decomposing the image into a plurality of spatial frequency bands;   extracting the one or more desired spatial frequency bands from the plurality of spatial frequency bands;   amplifying the one or more desired spatial frequency bands;   adding each band in the amplified one or more desired spatial frequency bands to a corresponding band in the plurality of spatial frequency bands to generate a plurality of augmented bands;   generating the augmented image by collapsing the plurality of augmented bands; and   displaying the augmented image on a display.   
     
     
         31 . The method of  claim 30 , wherein the one or more desired frequency bands are received via a user interface. 
     
     
         32 . The method of  claim 30 , wherein the one or more desired frequency bands are related to the region of interest. 
     
     
         33 . The method of  claim 30 , wherein the captured image is a video having a plurality of image frames. 
     
     
         34 . The method of  claim 33 , further comprising:
 producing an augmented video from augmented image frames.   
     
     
         35 . The method of  claim 30 , wherein decomposing the image is performed by a bandpass filter. 
     
     
         36 . The method of  claim 30 , wherein decomposing the image includes isolating the one or more desired spatial frequency bands from the plurality spatial frequency bands. 
     
     
         37 . The method of  claim 30 , further comprising:
 applying an edge detection algorithm to highlight one or more edges in the captured image; and   adding the one or more highlighted edges to the augmented image.   
     
     
         38 . The method of  claim 30 , further comprising:
 obtaining a plurality of hyper-spectral images;   combining the plurality of hyper-spectral images to generate a three dimensional hyper-spectral image cube; and   adding the three dimensional hyper-spectral image cube to the augmented image.   
     
     
         39 . The method of  claim 30 , further comprising:
 obtaining an infrared image; and   adding the infrared image to the augmented image.   
     
     
         40 . A non-transitory computer readable medium storing instructions that, when executed by a control device, cause the control device to perform a method comprising:
 capturing an image of the region of interest inside a patient during a surgical procedure;   receiving one or more desired frequency bands;   decomposing the image into a plurality of spatial frequency bands;   extracting the one or more desired spatial frequency bands from the plurality of spatial frequency bands;   amplifying the one or more desired spatial frequency bands;   adding each band in the amplified one or more desired spatial frequency bands to a corresponding band in the plurality of spatial frequency bands to generate a plurality of augmented bands;   generating the augmented image by collapsing the plurality of augmented bands; and   displaying the augmented image on a display.