Robotic systems and methods for navigation of luminal network that detect physiological noise

Abstract

Provided are robotic systems and methods for navigation of luminal network that detect physiological noise. In one aspect, the system includes a set of one or more processors configured to receive first and second image data from an image sensor located on an instrument, detect a set of one or more points of interest the first image data, and identify a set of first locations and a set of second location respectively corresponding to the set of points in the first and second image data. The set of processors are further configured to, based on the set of first locations and the set of second locations, detect a change of location of the instrument within a luminal network caused by movement of the luminal network relative to the instrument based on the set of first locations and the set of second locations.

What is claimed is: 
     
         1 . A medical robotic system, comprising:
 a set of one or more processors; and   at least one computer-readable memory in communication with the set of processors and having stored thereon computer-executable instructions to cause the set of processors to:
 receive first image data from an image sensor located on an instrument, the instrument configured to be driven through a luminal network of a patient, 
 detect a set of one or more points of interest the first image data, 
 identify a set of first locations respectively corresponding to the set of points in the first image data, 
 receive second image data from the image sensor, 
 detect the set of one or more points in the second image data, 
 identify a set of second locations respectively corresponding to the set of points in the second image data, and 
 based on the set of first locations and the set of second locations, detect a change of location of the instrument within the luminal network caused by movement of the luminal network relative to the instrument. 
   
     
     
         2 . The system of  claim 1 , wherein:
 the set of first locations and the set of second locations respectively define two-dimensional (2D) locations of the points within the first image data and the second image data.   
     
     
         3 . The system of  claim 2 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 group the set of points into a plurality of pairs of points, a first pair of points comprising a first point and a second point,   determine a first distance between the first point and the second point in the first image data based on the set of first locations, and   determine a second distance between the first point and the second point in the second image data based on the set of second locations, wherein detecting the change of location of the instrument within the luminal network is further based on the first distance and the second distance.   
     
     
         4 . The system of  claim 3 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 determine a first scale change estimate for the first pair of points based on the first distance and the second distance, and   determine a scale change value representative of the scale change between the first image data and the second image data based on the scale change estimate,   wherein detecting the change of location of the instrument within the luminal network is further based on the scale change value.   
     
     
         5 . The system of  claim 4 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 determine a set of scale change estimates respectively corresponding to the pairs of points, and   determine the scale change value based on a median value of the set of scale change estimates or an average value of the set of scale change estimates.   
     
     
         6 . The system of  claim 1 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 track cumulative changes to a scale change value representative of a scale change in image data received from the image sensor over a first time period,   transform the tracked scale change value into a frequency domain, and   identify at least one harmonic in the tracked scale change value in frequency domain,   wherein detecting the change of location of the instrument within the luminal network is further based on the at least one harmonic.   
     
     
         7 . The system of  claim 1 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 determine a location of the instrument based on data received from one or more location sensors,   determine a first physiological movement frequency of the patient based on the set of first locations and the set of second locations, wherein detecting the change of location of the instrument within the luminal network is further based on the first physiological movement frequency, and   provide an indication of the detected change of location of the instrument within the luminal network to a display.   
     
     
         8 . The system of  claim 7 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 determine a second physiological movement frequency of the patient based on the data received from the one or more location sensors, and   determine that a difference between the first physiological movement frequency and the second physiological movement frequency is less than a threshold difference,   wherein detecting the change of location of the instrument within the luminal network is further in response to determining that the difference between the first physiological movement frequency and the second physiological movement frequency is less than the threshold difference.   
     
     
         9 . The system of  claim 7 , wherein the one or more location sensors comprise at least one of: an electromagnetic (EM) sensor, a shape-sensing fiber, robot command data, and a radiation-based image sensor. 
     
     
         10 . The system of  claim 7 , wherein the physiological movement comprises at least one of a respiration of the patient or a heart rate of the patient. 
     
     
         11 . The system of  claim 1 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 identify a set of backtracked locations of the set of points in the first image data via backtracking the set of points from the second image data to the first image data,   compare the set of backtracked locations to the set of first locations,   identify a sub-set of points in the set of points for which the backtracked locations are not within a threshold distance of the set of first locations, and remove the sub-set of points from the set of points.   
     
     
         12 . The system of  claim 1 , wherein the first set of locations and the second set of locations comprise two-dimensional (2D) information indicative of the respective locations of the points with respect to a coordinate system of the first image data and the second image data. 
     
     
         13 . The system of  claim 1 , wherein the memory further has stored thereon computer-executable instructions to cause the set of processors to:
 extract depth information for the set of points from the first image data, and   extract depth information for the set of points from the second image data,   wherein the first set of locations and the second set of locations comprise three-dimensional (3D) information indicative of the respective locations of the points determined based on depth information extracted from each of the first image data and second image data.   
     
     
         14 . A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:
 receive first image data from an image sensor located on an instrument, the instrument configured to be driven through a luminal network of a patient;   detect a set of one or more points of interest the first image data;   identify a set of first locations respectively corresponding to the set of points in the first image data;   receive second image data from the image sensor;   detect the set of one or more points in the second image data;   identify a set of second locations respectively corresponding to the set of points in the second image data; and   based on the set of first locations and the set of second locations, detect a change of location of the instrument within the luminal network caused by movement of the luminal network relative to the instrument.   
     
     
         15 . The non-transitory computer readable storage medium of  claim 14 , wherein:
 the set of first locations and the set of second locations respectively define two-dimensional (2D) locations of the points within the first image data and the second image data.   
     
     
         16 . The non-transitory computer readable storage medium of  claim 15 , wherein the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause the at least one computing device to:
 group the set of points into a plurality of pairs of points, a first pair of points comprising a first point and a second point;   determine a first distance between the first point and the second point in the first image data based on the set of first locations; and   determine a second distance between the first point and the second point in the second image data based on the set of second locations,   wherein detecting the change of location of the instrument within the luminal network is further based on the first distance and the second distance.   
     
     
         17 . The non-transitory computer readable storage medium of  claim 16 , wherein the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause the at least one computing device to:
 determine a first scale change estimate for the first pair of points based on the first distance and the second distance; and   determine a scale change value representative of the scale change between the first image data and the second image data based on the scale change estimate,   wherein detecting the change of location of the instrument within the luminal network is further based on the scale change value.   
     
     
         18 . The non-transitory computer readable storage medium of  claim 17 , wherein the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause the at least one computing device to:
 determine a set of scale change estimates respectively corresponding to the pairs of points; and   determine the scale change value based on a median value of the set of scale change estimates or an average value of the set of scale change estimates.   
     
     
         19 . The non-transitory computer readable storage medium of  claim 14 , wherein the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause the at least one computing device to:
 track cumulative changes to a scale change value representative of a scale change in image data received from the image sensor over a first time period;   transform the tracked scale change value into a frequency domain; and   identify at least one harmonic in the tracked scale change value in frequency domain, wherein detecting the change of location of the instrument within the luminal network is further based on the at least one harmonic.   
     
     
         20 . A method for detecting a change of location of an instrument, comprising:
 receiving first image data from an image sensor located on the instrument, the instrument configured to be driven through a luminal network of a patient;   detecting a set of one or more points of interest the first image data;   identifying a set of first locations respectively corresponding to the set of points in the first image data;   receiving second image data from the image sensor;   detecting the set of one or more points in the second image data;   identifying a set of second locations respectively corresponding to the set of points in the second image data; and   based on the set of first locations and the set of second locations, detecting the change of location of the instrument within the luminal network caused by movement of the luminal network relative to the instrument.