Methods for planning and guiding endoscopic procedures that incorporate image-to-body divergence reduction

Abstract

A method for planning and guiding an endoscopic procedure, taking place in a hollow structure inside of a multi-organ cavity, that compensates for image-to-body divergence, first derives a procedure plan based on two scans of the multi-organ cavity at different volumes, which includes computing anatomical models for each scan, defining a region of interest (ROI) in the in the first scan, determining a route leading through the hollow structure to the ROI, deriving guidance cues that instruct how to maneuver the endoscope along the route toward the ROI, computing a 3D transformation that maps points in the first scan to corresponding points in the second scan, using the transformation to convert the route for the first scan into a corresponding route in the space of the second scan, and deriving guidance cues for the second scan that instruct how to maneuver the endoscope along the route toward the ROI.

The invention claimed is: 
     
         1 . A method for planning and guiding an endoscopic procedure taking place in a hollow structure inside of a multi-organ cavity, where the method compensates for image-to-body divergence, the method comprising the steps of:
 precomputing a procedure plan comprising the steps of:   a. receiving two 3D pre-operative imaging scans (I 1  and I 2 ) of the multi-organ cavity, wherein each scan depicts the multi-organ cavity at different states as a 3D space of I 1  and I 2  respectively;   b. computing anatomical models for each scan, with each model consisting of the shape and centerlines of the hollow structure and the shapes of organs inside the multi-organ cavity, wherein the centerlines consist of a series of connected ordered branches;   c. defining a region of interest (ROI) R in the 3D space of I 1 ;   d. computing a guidance route r 1 , which consists of a series of N≥0 connected ordered branches and a terminating segment, in the 3D space of I 1  inside the hollow structure to the ROI R leading to a localization pose, which instructs when to transition from endoscope navigation toward the ROI R to endoscope localization near the ROI R, and then terminating at a destination pose inside the hollow structure near the ROI R;   e. deriving guidance cues that instruct how to maneuver the endoscope along the guidance route r 1  toward the ROI R in the 3D space of I 1 ;   f. computing a 3D transformation T that maps points in the 3D space of I 1  to corresponding points in the 3D space of I 2 ;   g. using the 3D transformation T to convert the guidance route r 1  into a corresponding route r 2  in the 3D space of I 2 ;   h. deriving guidance cues for scan I 2  that instruct how to maneuver the endoscope along the route r 2  toward a destination pose near the ROI R in the 3D space of I 2 ;   performing a guidance protocol for guiding the endoscopic procedure comprising the steps of:   a. assigning the guidance route r 1  as an active guidance route r a  in the 3D space of I 1 ;   b. presenting preplanned guidance cues and virtual endoscope views along the active guidance route r a  in the 3D space of I 1 , which instruct the endoscope operator on how to maneuver the real endoscope along the active guidance route r a  through the hollow structure toward the predefined ROI R;   c. repeating step b) a plurality of times until reaching the preplanned localization pose;   d. switching the active guidance route r a  to the route r 2 ;   e. presenting preplanned guidance cues and virtual endoscope views along the active guidance route r a  in the 3D space of I 2 , which instruct the endoscope operator on how to make final real endoscope maneuvers along the active guidance route r a  through the hollow structure to reach the preplanned destination pose near the ROI R;   f. repeating step e) a plurality of times until reaching the preplanned destination pose; whereby the 3D poses of the real endoscope and virtual endoscope are synchronized at a final destination pose for examining the ROI R.   
     
     
         2 . The method of  claim 1 , wherein the precomputation of the procedure plan further comprises:
 deriving a set of back-up procedure plans, which comprises the steps of:
 a. constructing a partial guidance route {circumflex over (r)} 1   n  for each n≤N, if possible, starting at n=1, defined by the first n branches of the guidance route r 1 ; 
 b. deriving a back-up procedure plan for each n≤N, if possible, starting at n=1, the back-up procedure plan including: 1) a back-up guidance route r 1   n  inside the hollow structure, wherein the back-up guidance route r 1   n  consists of partial guidance route {circumflex over (r)} 1   n  concatenated with other connected order branches and a terminating segment leading to a localization pose, which instructs when to transition from endoscope navigation toward the ROI R to endoscope localization near the ROI R, and terminating at a destination pose inside the hollow structure near the ROI R; and 2) guidance cues, which instruct how to maneuver the endoscope along the back-up guidance route r 1   n ; 
 c. computing a back-up guidance route r 2   n  for each n≤N, if possible, starting at n=1, in the 3D space of I 2  by using the transformation T to map the back-up guidance route r 1   n  into the 3D space of I 2 ; 
 d. modifying the back-up guidance route r 2   n  for each n≤N, if possible, starting at n=1, such that it terminates at a destination pose inside the hollow structure near the ROI R in the 3D space of I 2 ; 
 e. deriving guidance cues in the 3D space of I 2  indicating how to maneuver the endoscope along the back-up guidance route r 2   n  for each n≤N, if possible, starting at n=1, toward the destination pose; and 
   wherein the step of performing the guidance protocol further comprises:
 providing guidance information that enables the endoscope operator to switch the active guidance route r a  to the back-up guidance route r 1   n  upon reaching the end of the n th  branch of r a , n<N, if available, and continue navigating the endoscope to the destination pose of the back-up guidance route r 1   n  near the ROI R, which comprises the steps of: 
 i. signaling the endoscope operator that the back-up guidance route r 1   n  is available; 
 ii. when the endoscope operator chooses to switch to the back-up guidance route r 1   n , assigning the back-up guidance route r 1   n  as the active guidance route r a , updating the endoscope's position and distance to the destination pose, based on the active guidance route r a , and providing guidance cues associated with the active guidance route r a . 
   
     
     
         3 . The method of  claim 1 , wherein the multi-organ cavity is a thoracic cavity, including the mediastinum, an abdominal cavity, a pelvic cavity, or a cranial cavity. 
     
     
         4 . The method of  claim 1 , wherein the hollow structure is the lung airways, colon, stomach, vasculature of the heart, or any other hollow regions situated inside of a multi-organ cavity. 
     
     
         5 . The method of  claim 1 , wherein the endoscope is bronchoscope, laparoscope, colonoscope, cystoscope, or cardiac catheter. 
     
     
         6 . The method of  claim 1 , wherein the method receives endoscope tip location and insertion depth information from a robotics assisted system during the live endoscopic procedure for the purpose of updating the distance of the virtual endoscope to the destination pose. 
     
     
         7 . The method of  claim 1 , wherein the 3D pre-operative imaging scan I 1  is a CT scan or MRI scan. 
     
     
         8 . The method of  claim 1 , wherein the 3D pre-operative imaging scan I 2  is a CT scan, a CT scan derived from a PET/CT study, an MRI scan, or a simulated scan generated from I 1 . 
     
     
         9 . The method of  claim 1 , wherein the ROI R is a suspect tumor, cancer nodule, suspicious airway wall site, or an anatomical location of interest for general visual examination, tissue biopsy, or treatment delivery. 
     
     
         10 . The method of  claim 1 , wherein examination of the ROI R involves collecting anatomical tissue from the ROI R, performing a visual assessment of the ROI R, or delivering treatment to the ROI R. 
     
     
         11 . The method of  claim 1 , wherein the endoscope operator is a physician or a robot. 
     
     
         12 . The method of  claim 1 , wherein, when the guidance protocol provides guidance information that enables the endoscope operator to localize the endoscope at a destination pose near ROI R in the 3D space of I 2 , the guidance protocol continues to display guidance information in both spaces simultaneously as guidance progresses. 
     
     
         13 . A system for planning and guiding an endoscopic procedure taking place in a hollow structure inside a patient's multi-organ cavity, where the system compensates for image-to-body divergence, the system comprising:
 a processing device; and   a non-transitory, processor-readable storage medium comprising one or more programming instructions stored thereon that, when executed, cause the processing device to:   precompute a procedure plan comprising:   a. receive two 3D pre-operative imaging scans (I 1  and I 2 ) of the multi-organ cavity, wherein each scan depicts the multi-organ cavity at different states as a 3D space of I 1  and I 2  respectively;   b. compute anatomical models for each scan, with each model consisting of the shape and centerlines of the hollow structure and the shapes of organs inside the multi-organ cavity, wherein the centerlines consist of a series of connected ordered branches;   c. define a region of interest (ROI) R in the 3D space of I 1 ;   d. compute a guidance route r 1 , which consists of a series of N≥0 connected ordered branches and a terminating segment, in the 3D space of I 1  inside the hollow structure to the ROI R leading to a localization pose, which instructs when to transition from endoscope navigation toward the ROI R to endoscope localization near the ROI R, and then terminating at a destination pose inside the hollow structure near the ROI R;   e. derive guidance cues that instruct how to maneuver the endoscope along the guidance route r 1  toward the ROI R in the 3D space of I 1 ;   f. compute a 3D transformation T that maps points in the 3D space of I 1  to corresponding points in the 3D space of I 2 ;   g. use the 3D transformation T to convert the guidance route r 1  into a corresponding route r 2  in the 3D space of I 2 ;   h. derive guidance cues for scan I 2  that instruct how to maneuver the endoscope along the route r 2  toward a destination pose near the ROI R in the 3D space of I 2 ; and   perform a guidance protocol for guiding the endoscopic procedure comprising:   a. assign the guidance route r 1  as an active guidance route r a  in the 3D space of I 1 ;   b. present preplanned guidance cues and virtual endoscope views along the active guidance route r a  in the 3D space of I 1 , which instruct the endoscope operator on how to maneuver the real endoscope along the active guidance route r a  through the hollow structure toward the predefined ROI R;   c. repeat step b) a plurality of times until reaching the preplanned localization pose;   d. switch the active guidance route r a  to the route r 2 ;   e. present preplanned guidance cues and virtual endoscope views along the active guidance route r a  in the 3D space of I 2 , which instruct the endoscope operator on how to make final real endoscope maneuvers along the active guidance route r a  through the hollow structure to reach the preplanned destination pose near the ROI R;   f. repeat step e) a plurality of times until reaching the preplanned destination pose; whereby the 3D poses of the real endoscope and virtual endoscope are synchronized at a final destination pose for examining the ROI R.   
     
     
         14 . The system according to  claim 13 , wherein the one or more programming instructions stored thereon, when executed, further cause the processing device to:
 further precompute the procedure plan including:   derive a set of back-up procedure plans, which comprises the steps of:
 a. constructing a partial guidance route {circumflex over (r)} 1   n  for each n≤N, if possible, starting at n=1, defined by the first n branches of the guidance route r 1 ; 
 b. deriving a back-up procedure plan for each n≤N, if possible, starting at n=1, the back-up procedure plan including: 1) a back-up guidance route r 1   n  inside the hollow structure, wherein the back-up guidance route r 1   n  consists of partial guidance route {circumflex over (r)} 1   n  concatenated with other connected order branches and a terminating segment leading to a localization pose, which instructs when to transition from endoscope navigation toward the ROI R to endoscope localization near the ROI R, and terminating at a destination pose inside the hollow structure near the ROI R; and 2) guidance cues, which instruct how to maneuver the endoscope along the guidance route r 1   n ; 
 c. computing a back-up guidance route r 2   n  for each n≤N, if possible, starting at n=1, in the 3D space of I 2  by using the transformation T to map the back-up guidance route r 1   n  into the 3D space of I 2 ; 
 d. modifying the back-up route r 2   n  for each n≤N, if possible, starting at n=1, such that it terminates at a destination pose inside the hollow structure near the ROI R in the 3D space of I 2 ; 
 e. deriving guidance cues in the 3D space of I 2  indicating how to maneuver the endoscope along the back-up guidance route r 2   n  for each n≤N, if possible, starting at n=1, toward the destination pose; and 
   perform the guidance protocol further including:
 provide guidance information that enables the endoscope operator to switch the active guidance route r a  to the back-up guidance route r 1   n  upon reaching the end of the n th  branch of the active guidance route r a , n≤N, if available, and continue navigating the endoscope to the destination pose of route r 1   n  near the ROI R, which comprises the steps of:
 i. signal the endoscope operator that the back-up guidance route r 1   n  is available; 
 ii. when the endoscope operator chooses to switch to the back-up guidance route r 1   n , assign the back-up guidance route r 1   n  as the active guidance route r a , update the endoscope's position and distance to the destination pose, based on the active guidance route r a , and provide guidance cues associated with the active guidance route r a . 
 
   
     
     
         15 . The system of  claim 13 , wherein the multi-organ cavity is a thoracic cavity, including the mediastinum, an abdominal cavity, a pelvic cavity, or a cranial cavity. 
     
     
         16 . The system of  claim 13 , wherein the hollow structure is the lung airways, colon, stomach, vasculature of the heart, or any other hollow regions situated inside of a multi-organ cavity. 
     
     
         17 . The system of  claim 13 , wherein the endoscope is bronchoscope, laparoscope, colonoscope, cystoscope, or cardiac catheter. 
     
     
         18 . The system of  claim 13 , wherein the method receives endoscope tip location and insertion depth information from a robotics assisted system during the live endoscopic procedure for the purpose of updating the distance of the virtual endoscope to the destination pose. 
     
     
         19 . The system of  claim 13 , wherein the 3D pre-operative imaging scan I 1  is a CT scan or MRI scan. 
     
     
         20 . The system of  claim 13 , wherein the 3D pre-operative imaging scan I 2  is a CT scan, a CT scan derived from a PET/CT study, an MRI scan, or a simulated scan generated from I 1 . 
     
     
         21 . The system of  claim 13 , wherein the ROI R is a suspect tumor, cancer nodule, suspicious airway wall site, or an anatomical location of interest for general visual examination, tissue biopsy, or treatment delivery. 
     
     
         22 . The system of  claim 13 , wherein examination of the ROI R involves collecting anatomical tissue from the ROI R, performing a visual assessment of the ROI R, or delivering treatment to the ROI R. 
     
     
         23 . The system of  claim 13 , wherein the endoscope operator is a physician or a robot. 
     
     
         24 . The system of  claim 13 , wherein, when the guidance protocol provides guidance information that enables the endoscope operator to localize the endoscope at a destination pose near ROI R in the 3D space of I 2 , the guidance protocol continues to display guidance information in both spaces simultaneously as guidance progresses.