Image-Guided Surgery System

Abstract

An image-guided surgical system includes a processor, a display communicatively coupled to the processor, and an imaging system communicatively coupled to the processor. A memory device, communicatively coupled to the processor, stores instructions, executable by the processor, to cause the processor to receive, from the imaging system, real-time image data of an ophthalmological surgical field during an ophthalmological surgical procedure, and analyze the image data in real-time to identify an ocular tissue boundary present in the image data of the ophthalmological surgical field. The instructions cause the processor to provide real-time visual, auditory, and/or haptic feedback in response to the identified ocular tissue boundary.

1 . A surgical guidance system comprising:
 a processor;   a display communicatively coupled to the processor;   an imaging system communicatively coupled to the processor;   a memory device, communicatively coupled to the processor, and storing instructions, executable by the processor, to cause the processor to:
 receive, from the imaging system, real-time image data of an ophthalmological surgical field during an ophthalmological surgical procedure; 
 analyze the image data in real-time to identify an ocular tissue boundary present in the image data of the ophthalmological surgical field; and 
 provide real-time visual, auditory, and/or haptic feedback in response to the identified ocular tissue boundary. 
   
     
     
         2 . The surgical guidance system of  claim 1 , wherein the instructions stored on the memory device are further executable to cause the processor to:
 receive data from which a position of a tip of a surgical instrument may be determined, the received data comprising at least one of receiving the real-time image data from the imaging system and receiving sensor data from one or more sensors physically coupled to the surgical instrument; and   determine, in real-time, a distance between a tip of the surgical instrument and the ocular tissue boundary and   wherein providing real-time visual, auditory, and/or haptic feedback in response to the identified tissue boundary comprises providing the real-time visual, auditory, and/or haptic feedback in response to the distance between the tip of the surgical instrument and the ocular tissue boundary.   
     
     
         3 . The surgical guidance system of  claim 2 , wherein receiving data from which the position of the tip of the surgical instrument may be determined comprises receiving the real-time image data from the imaging system. 
     
     
         4 . The surgical guidance system of  claim 3 , wherein the instructions are further executable by the processor to analyze, in real-time, the image data received from the imaging system to identify in the surgical field the tip of the surgical instrument. 
     
     
         5 . The surgical guidance system of  claim 4 , wherein identifying in the surgical field the tip of the surgical instrument comprises inputting the image data into a trained artificial intelligence (AI) model configured to identify the tip of the surgical instrument based on the image data. 
     
     
         6 . The surgical guidance system of  claim 1 , wherein:
 the imaging system comprises a digital stereoscopic vision system;   the surgical guidance system is a microsurgical guidance system;   the surgical field is a microsurgical field; and   the surgical instrument is a microsurgical instrument.   
     
     
         7 . The surgical guidance system of  claim 1 , wherein analyzing the image data in real-time to identify an ocular tissue boundary present in the image data of the surgical field comprises inputting the image data into a trained artificial intelligence (AI) model configured to identify ocular tissue boundaries and/or tissue types based on the image data. 
     
     
         8 . The surgical guidance system of  claim 7 , wherein inputting the image data into a trained AI model configured to identify tissue boundaries and/or tissue types comprises inputting into the trained AI model configured to identify tissue boundaries and/or tissue types image data on which one or more of the following processes has been implemented: compression, cropping, filtering, normalization, contrast adjustment, brightness adjustment, conversion to greyscale, downsampling, and/or upsampling. 
     
     
         9 . The surgical guidance system of  claim 1 , wherein the system provides real-time visual feedback in response to the identified ocular tissue boundary, and wherein providing real-time visual feedback comprises one or more of:
 identifying on a displayed image of the surgical field a tip of a surgical instrument relative to the identified ocular tissue boundary;   changing a color associated with the tip of the surgical instrument according to a distance between the tip of the surgical instrument and the identified ocular tissue boundary;   changing a color associated with the identified ocular tissue boundary in response to a determined parameter associated with the identified ocular tissue boundary;   changing a brightness associated with the tip of the surgical instrument according to the distance between the tip of the surgical instrument and the tissue boundary;   changing a brightness associated with the identified ocular tissue boundary in response to a determined parameter associated with the identified ocular tissue boundary;   placing text on a displayed image of the surgical field to warn of proximity of the surgical instrument to the tissue boundary;   placing text on a displayed image in response to the determined parameter associated with the identified ocular tissue boundary; or   augmenting a displayed image of the surgical field to highlight the tissue or the tissue boundary.   
     
     
         10 . The surgical guidance system of  claim 1 , wherein the system provides real-time auditory feedback in response to the identified ocular issue boundary, and wherein providing real-time auditory feedback comprises one or more of:
 annunciating a verbal warning that a tip of a surgical instrument is approaching the identified ocular tissue boundary;   providing an audible tone or alarm in response to the proximity of the tip of the surgical instrument to the tissue boundary;   changing a pitch of an audible tone or alarm in response to the proximity of the tip of the surgical instrument to the tissue boundary;   changing a volume of an audible tone or alarm in response to the proximity of the tip of the surgical instrument to the tissue boundary;   annunciating a verbal warning that a determined parameter associated with the identified ocular tissue boundary is approaching or exceeding a threshold;   providing an audible tone or alarm in response to the determined parameter associated with the identified ocular tissue boundary approaching or exceeding a threshold;   changing a pitch of an audible tone or alarm in response to the determined parameter associated with the identified ocular tissue boundary approaching or exceeding a threshold; or   changing a volume of an audible tone or alarm in response to the determined parameter associated with the identified ocular tissue boundary approaching or exceeding a threshold.   
     
     
         11 . The surgical guidance system of  claim 1 , wherein the system provides real-time haptic feedback in response to the identified ocular issue boundary, the system further comprising a haptic device, wherein providing real-time haptic feedback comprises one or more of:
 vibrating the haptic device as the tip of the surgical instrument crosses a threshold distance from the tissue boundary;   causing the haptic device to increase resistance to movement of the tip of the surgical instrument in the direction of the tissue boundary as the tip of the surgical instrument approaches the tissue boundary; or   causing, in a haptic device that is robotically controlling the movement of the surgical device, the haptic device to change the ratio of movement of the haptic device to movement of the tip of the surgical instrument as the tip of the surgical instrument approaches the tissue boundary;   vibrating the haptic device as a parameter associated with the identified ocular tissue boundary approaches or exceeds a threshold;   causing the haptic device to increase resistance to movement of a surgical instrument as a parameter associated with the identified ocular tissue boundary approaches or exceeds a threshold; or   causing, in a haptic device that is robotically controlling the movement of a surgical instrument, the haptic device to (1) change the ratio of movement of the haptic device to movement of the surgical instrument as a parameter associated with the identified ocular tissue boundary approaches or exceeds a threshold, or (2) withdraw the surgical instrument from the surgical field if collision with tissue is imminent.   
     
     
         12 . The surgical guidance system of  claim 1 , further comprising displaying a parameter and/or a warning related to the deformation or compression of tissue in the surgical field, the parameter or warning determined according to the real-time image data. 
     
     
         13 . The surgical guidance system of  claim 12 , wherein the parameter and/or warning are associated with one or more of: a change in volume of an entity in the surgical field; a change in area of an entity in the surgical field; a movement of a tissue in the surgical field; and a calculated shear stress applied to a tissue in the surgical field. 
     
     
         14 . A method of providing real-time feedback to a surgeon during a microsurgical procedure, the method comprising:
 receiving real-time image data from an imaging system communicatively coupled to a processor;   performing, in the processor, in real-time, segmentation of the image data to determine one or more tissue boundaries;   performing, in the processor, in real-time, identification of a position of a tip of a microsurgical instrument relative to one of the one or more tissue boundaries;   providing real-time visual, auditory, and/or haptic feedback to the surgeon based on the predetermined distance between the tip of the microsurgical instrument and the one of the one or more tissue boundaries.   
     
     
         15 . A microsurgical guidance system comprising:
 a haptic device configured to provide direct feedback during a microsurgical procedure from different sources to a microsurgical instrument in real-time.   
     
     
         16 . The microsurgical guidance system of  claim 15 , the system configured to receive real-time imaging data from an imaging and/or video capture device, the system configured to analyze and deliver direct modulation feedback concerning a position and/or function of the microsurgical instrumentation, and comprising:
 a display operatively connected to the haptic device and the microsurgical instrument; and   a feedback loop, whereby the location of the microsurgical instrument in relation to delicate ocular tissues is determined and the effect of instrument-tissue interactions is used in the analysis to guide surgical maneuvers with direct feedback;   wherein the feedback loop is operably connected to the imaging and/or video capture device, the microsurgical instrument, and the haptic device, and   wherein the feedback loop generates an output including a tactile feedback signal, the output delivered to the haptic device, the output including graded resistance to movement of the microsurgical instrument approaching a threshold and/or exclusion zone defined as proximity to tissues or structures to be avoided.   
     
     
         17 . A method of guiding a microsurgical instrument comprising:
 receiving image data in real-time from an imaging system operably connected, directly or indirectly, to the microsurgical instrument;   segmenting and selecting slices or subsections from the image data and correlating the resulting image data with instrument position data via a processor operably connectable, directly or indirectly, to the microsurgical instrument to produce an output image, where the instrument position data can be acquired, at least in part, from a direct haptic feedback device;   generating feedback in real-time via a feedback loop operably connected to the processor;   sending the feedback to the microsurgical instrument via the direct haptic feedback device; and   controlling the power of the microsurgical instrument, if needed.   
     
     
         18 . A microsurgical guidance system comprising:
 an imaging device configured to acquire tissue surround data in real-time;   a haptic feedback device configured to acquire and send instrument position data derived from a surgical instrument; and   a feedback loop configured to correlate tissue surround data and instrument position data, the feedback loop configured to analyze the tissue surround data and instrument position data to form an analyzed data output, the feedback loop configured to generate at least one of a display output, an audible output, and a haptic feedback output based on the analyzed data output.   
     
     
         19 . The system of  claim 18 , the feedback loop configured to identify tissue planes and/or anatomical structures of the real-time anatomical images from the imaging device output. 
     
     
         20 . In systems configured to guide microsurgical procedures, a method of expediting feedback to the surgeon in real-time, the method comprising:
 reducing a volume of an imaged area, if needed;   performing image processing on a selected portion of the imaged area;   targeting imaging processing of tissue boundaries while excluding internal structure during the imaging processing; and   analyzing the targeted portion in view of anatomical changes to the images area and microsurgical instrument during the procedure.   
     
     
         21 . A system for guiding microsurgical procedures, the system comprising:
 a microprocessor;   an imaging system coupled to the microprocessor;   a memory device coupled to the microprocessor, the memory device storing machine readable instructions that, when executed by the processor, cause the processor to:
 receive from the imaging system image data of an imaged area; 
 reduce the volume of the imaged area, if needed; 
 perform image processing on a selected portion of the imaged area 
 target image processing of tissue boundaries while excluding internal structure during the image processing; and 
 analyze the targeted portion in view of anatomical changes to the image area and microsurgical instrument during the procedure. 
   
     
     
         22 . A memory device, communicatively coupled to a processor, and storing instructions, executable by the processor, to cause the processor to:
 receive, from an imaging device communicatively coupled to the processor, real-time image data of a surgical field during a surgical procedure;   analyze the image data in real-time to identify a tissue boundary present in the image data of the surgical field; and   provide real-time visual, auditory, and/or haptic feedback in response to the identified tissue boundary.   
     
     
         23 . An artificial intelligence (AI)-based surgical guidance system configured to provide real-time feedback to a surgeon during a surgical procedure, the system comprising:
 a first trained AI model configured to receive real-time images of a surgical field and to identify, in real-time, within the images, one or more tissue boundaries;   an analysis module configured to:
 receive, from the first trained AI model, data indicating the one or more tissue boundaries identified by the first trained AI model; 
 receive data indicating the position of the tip of the surgical instrument; 
 determine, in real-time, from the data received from the first trained AI model and the data indicating the position of the tip of the surgical instrument, a distance between the tip of the surgical instrument and one of the one or more tissue boundaries; 
 provide real-time visual, auditory, and/or haptic feedback to the surgeon based on the determined distance between the tip of the surgical instrument and the one of the one or more tissue boundaries.