System and method for identifying, marking and navigating to a target using real time two dimensional fluoroscopic data

Abstract

A system for facilitating identification and marking of a target in a fluoroscopic image of a body region of a patient, the system comprising one or more storage devices having stored thereon instructions for: receiving a CT scan and a fluoroscopic 3D reconstruction of the body region of the patient, wherein the CT scan includes a marking of the target; and generating at least one virtual fluoroscopy image based on the CT scan of the patient, wherein the virtual fluoroscopy image includes the target and the marking of the target, at least one hardware processor configured to execute these instructions, and a display configured to display to a user the virtual fluoroscopy image and the fluoroscopic 3D reconstruction.

What is claimed is: 
     
       1. A system for facilitating identification and marking of a target in a target area in a fluoroscopic image of a body region of a patient, the system comprising:
 (i) one or more storage devices having stored thereon instructions for:
 receiving a CT scan of the body region of the patient, wherein the CT scan includes a marking of the target; 
 receiving a sequence of fluoroscopic images including the target area acquired in real-time about a plurality of angles relative to the target, while a medical device is positioned in proximity to the target; and 
 generating a fluoroscopic 3D reconstruction based on at least a portion of the sequence of fluoroscopic images; 
 generating at least one virtual fluoroscopic image based on the CT scan of the patient, wherein the virtual fluoroscopic image includes the target and the marking of the target; 
 receiving a selection of the target in the fluoroscopic 3D reconstruction via a user input; 
 receiving a selection of the medical device in the fluoroscopic 3D reconstruction or the sequence of fluoroscopic images; 
 determining an offset of the medical device with respect to the target based on the selections of the target and the medical device; 
 determining a location of the medical device within the patient based on data provided by a tracking system; 
 displaying the target area and the location of the medical device with respect to the target on a display; and 
 correcting the display of the location of the medical device with respect to the target based on the offset between the medical device and the target; and 
 
 (ii) at least one hardware processor configured to execute said instructions. 
 
     
     
       2. The system of  claim 1 , wherein the one or more storage devices have stored thereon further instructions for directing the user to identify and mark the target in the fluoroscopic 3D reconstruction while using the virtual fluoroscopic image as a reference. 
     
     
       3. The system of  claim 1 , wherein the user is directed to identify and mark the target in two fluoroscopic slice images of the fluoroscopic 3D reconstruction. 
     
     
       4. The system of  claim 1 , wherein the generating of the at least one virtual fluoroscopic image comprises:
 generating virtual fluoroscope poses around the target by simulating a fluoroscope trajectory while scanning the target; 
 generating virtual fluoroscopic images by projecting the CT scan according to the virtual fluoroscope poses; 
 generating a virtual fluoroscopic 3D reconstruction based on the virtual fluoroscopic images; and 
 selecting a slice image from the virtual fluoroscopic 3D reconstruction which comprises the marking of the target. 
 
     
     
       5. The system of  claim 1 , wherein the target is a soft-tissue target. 
     
     
       6. The system of  claim 1 , wherein the one or more storage devices have stored thereon further instructions for:
 generating a 3D rendering of the target area based on the CT scan, wherein the display of the target area comprises displaying the 3D rendering; and 
 registering data generated by the tracking system to the 3D rendering, wherein correcting the display of the location of the medical device with respect to the target comprises updating the registration between the data generated by the tracking system and the 3D rendering or locally registering the fluoroscopic 3D reconstruction to the 3D rendering. 
 
     
     
       7. The system of  claim 1 , wherein the target area comprises at least a portion of lungs and the medical device is configured to be navigated to the target area through a luminal network. 
     
     
       8. A method for identifying and marking a target in a target area in an image of a body region of a patient, the method comprising using at least one hardware processor for:
 receiving a CT scan of the body region of the patient, wherein the CT scan includes a marking of the target; 
 receiving a sequence of fluoroscopic images of the target area acquired in real-time about a plurality of angles relative to the target area, while a medical device is positioned in the target area; 
 generating a fluoroscopic 3D reconstruction of the target area based on at least a portion of the sequence of fluoroscopic images; 
 generating at least one virtual fluoroscopic image based on the CT scan of the patient, wherein the at least one virtual fluoroscopic image includes the target and the marking of the target; 
 displaying to a user the at least one virtual fluoroscopic image and the fluoroscopic 3D reconstruction on a display, thereby facilitating the identification of the target in the fluoroscopic 3D reconstruction by the user; 
 receiving a selection of the target in the fluoroscopic 3D reconstruction via a user input; 
 receiving a selection of the medical device in the fluoroscopic 3D reconstruction or the sequence of fluoroscopic images; 
 determining an offset of the medical device with respect to the target based on the selections of the target and the medical device; 
 determining a location of the medical device within the patient based on data provided by a tracking system; 
 displaying the target area and the location of the medical device with respect to the target on a display; and 
 correcting the display of the location of the medical device with respect to the target based on the determined offset between the medical device and the target. 
 
     
     
       9. The method of  claim 8 , further comprising using said at least one hardware processor for directing the user to identify and mark the target in the fluoroscopic 3D reconstruction while using the at least one virtual fluoroscopic image as a reference. 
     
     
       10. The method of  claim 8 , further comprising using said at least one hardware processor for instructing the user to identify and mark the target in two fluoroscopic slice images of the fluoroscopic 3D reconstruction. 
     
     
       11. The method of  claim 8 , wherein the generating of the at least one virtual fluoroscopic image comprises:
 generating virtual fluoroscope poses around the target by simulating a fluoroscope trajectory while scanning the target; 
 generating virtual fluoroscopic images by projecting the CT scan according to the virtual fluoroscope poses; 
 generating a virtual fluoroscopic 3D reconstruction based on the virtual fluoroscopic images; and 
 selecting a slice image from the virtual fluoroscopic 3D reconstruction which comprises the marking of the target. 
 
     
     
       12. The method of  claim 8 , wherein the target is a soft-tissue target. 
     
     
       13. The method of  claim 8 , further comprising using said at least one hardware processor for:
 generating a 3D rendering of the target area based on the CT scan, wherein the displaying of the target area comprises displaying the 3D rendering; and 
 registering data generated by the tracking system to the 3D rendering, wherein correcting the display of the location of the medical device with respect to the target comprises updating the registration between the data generated by the tracking system and the 3D rendering or locally registering the fluoroscopic 3D reconstruction to the 3D rendering. 
 
     
     
       14. The method of  claim 8 , wherein a structure of markers is placed with respect to the patient and a fluoroscopic imaging device configured to acquire the sequence of fluoroscopic images such that each image of the sequence of fluoroscopic images comprises a projection of at least a portion of the structure of markers, and wherein generating of the fluoroscopic 3D reconstruction of the target area comprises estimating a pose of the fluoroscopic imaging device while acquiring each image of the sequence of fluoroscopic images based on detection of a possible and most probable projection of the structure of markers as a whole on each image. 
     
     
       15. The method of  claim 8 , wherein the at least one virtual fluoroscopic image and the fluoroscopic 3D reconstruction are displayed simultaneously. 
     
     
       16. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:
 receive a pre-operative CT scan of a target area, wherein the pre-operative CT scan includes a marking of the target; 
 generate at least one virtual fluoroscopic image based on the pre-operative CT scan, wherein the at least one virtual fluoroscopic image includes the target and the marking of the target; 
 receive a sequence of fluoroscopic images of the target area acquired in real-time about a plurality of angles relative to the target area, while a medical device is positioned in the target area; 
 generate a fluoroscopic three-dimensional reconstruction of the target area based on the sequence of fluoroscopic images; 
 display to a user the at least one virtual fluoroscopic image and the fluoroscopic three-dimensional reconstruction, 
 receive a selection of the target in the fluoroscopic three-dimensional reconstruction via a user input; 
 receive a selection of the medical device in the fluoroscopic three-dimensional reconstruction or the sequence of fluoroscopic images; and 
 determine an offset of the medical device with respect to the target based on the selections of the target and the medical device.