Computer assisted identification of appropriate anatomical structure for medical device placement during a surgical procedure

Abstract

A method for computer assisted identification of appropriate anatomical structure for placement of a medical device, comprising: receiving a 3D scan volume comprising set of medical scan images of a region of an anatomical structure where the medical device is to be placed; automatically processing the set of medical scan images to perform automatic segmentation of the anatomical structure; automatically determining a subsection of the 3D scan volume as a 3D ROI by combining the raw medical scan images and the obtained segmentation data; automatically processing the ROI to determine the preferred 3D position and orientation of the medical device to be placed with respect to the anatomical structure by identifying landmarks within the anatomical structure with a pre-trained prediction neural network; automatically determining the preferred 3D position and orientation of the medical device to be placed with respect to the 3D scan volume of the anatomical structure.

1 .- 7 . (canceled) 
     
     
         8 . A method, comprising:
 receiving a set of three-dimensional (3D) scan volumes each including image data of an anatomical structure with a medical device implanted in the anatomical structure;   processing each of the set of 3D scan volumes to obtain segmentation data that identifies different anatomical parts of the anatomical structure of each of the set of 3D scan volumes;   merging, for each of the set of 3D scan volumes, the segmentation data of that 3D scan volume with the image data of that 3D scan volume to produce combined image data for that 3D scan volume;   identifying a 3D region of interest (ROI) in the combined image data of each of the set of 3D scan volumes that includes a subsection of the anatomical structure of that 3D scan volume with the medical device implanted therein;   generating a training database including (1) the 3D ROI identified for each of the set of 3D scan volumes and (2) marked characteristic features associated with a set of landmarks in the 3D ROI identified for each of the set of 3D scan volumes, the set of landmarks including an anatomical part of the anatomical structure and a portion of the medical device implanted in the anatomical structure; and   training, using the training database, a prediction neural network model to identify preferred positions and orientations for placement of the medical device in the anatomical structure.   
     
     
         9 . The method of  claim 8 , further comprising:
 augmenting the training database by: transforming the 3D ROI identified for each of the set of 3D scan volumes using one or more of: rotation, scaling, movement, horizontal flip, additive noise of Gaussian or Poisson distributions and Gaussian blur, volumetric grid deformation, or a generative algorithm, and generating an augmented training database including the 3D ROIs and the transformed 3D ROIs; and   training the prediction neural network model using the augmented training database.   
     
     
         10 . The method of  claim 9 , wherein the augmenting the training database further includes recalculating a position of the set of landmarks in each of the transformed 3D ROIs and associating the recalculated position of the set of landmarks with the transformed 3D ROIs. 
     
     
         11 . The method of  claim 8 , further comprising:
 resizing, for each of the set of 3D scan volumes, the 3D ROI identified for that 3D scan volume such that each 3D ROI has the same size,   the training database including the resized 3D ROI for each of the set of 3D scan volumes and the marked characteristic features associated with the set of landmarks in the resized 3D ROI.   
     
     
         12 . The method of  claim 8 , further comprising determining, using the trained prediction neural network model, a preferred placement of the medical device in an anatomical structure of a patient. 
     
     
         13 . The method of  claim 12 , wherein the determining the preferred placement of the medical device in the anatomical structure of the patient includes:
 determining, based on a 3D scan volume of anatomical structure of the patient and segmentation data of the anatomical structure of the patient, a 3D ROI in the 3D scan volume of the anatomical structure of the patient for placement of the medical device;   processing, using the trained prediction neural network model, the 3D ROI of the patient to identify the set of landmarks in the 3D ROI of the patient; and   determining a 3D position and orientation of the medical device for the preferred placement of the medical device based on the set of landmarks.   
     
     
         14 . The method of  claim 13 , further comprising visualizing the 3D position and orientation of the medical device for the preferred placement of the medical device with respect to the anatomical structure. 
     
     
         15 . The method of  claim 13 , further comprising
 identifying, after the medical device has been placed in the anatomical structure of the patient, an actual 3D position and orientation of the medical device; and   further training the prediction neural network model based on the actual 3D position and orientation of the medical device.   
     
     
         16 . The method of  claim 8 , wherein the anatomical part of the set of landmarks includes a pedicle, and the portion of the medical device of the set of landmarks includes a tip of the medical device. 
     
     
         17 . A method, comprising:
 receiving a set of three-dimensional (3D) scan volumes each including image data of an anatomical structure with a medical device implanted in the anatomical structure;   processing each of the set of 3D scan volumes to obtain segmentation data that identifies different anatomical parts of the anatomical structure of each of the set of 3D scan volumes;   identifying a 3D region of interest (ROI) in each of the set of 3D scan volumes that includes a subsection of the anatomical structure of that 3D scan volume with the medical device implanted therein;   generating a training database including (1) the 3D ROI identified for each of the set of 3D scan volumes and (2) marked characteristic features associated with a set of landmarks in the 3D ROI identified for each of the set of 3D scan volumes, the set of landmarks including an anatomical part of the anatomical structure and a portion of the medical device implanted in the anatomical structure;   training, using the training database, a prediction neural network model to identify preferred positions and orientations for placement of the medical device in the anatomical structure; and   determining, using the trained prediction neural network model, a preferred placement of the medical device in an anatomical structure of a patient.   
     
     
         18 . The method of  claim 17 , further comprising:
 augmenting the training database by: transforming the 3D ROI identified for each of the set of 3D scan volumes using one or more of: rotation, scaling, movement, horizontal flip, additive noise of Gaussian or Poisson distributions and Gaussian blur, volumetric grid deformation, or a generative algorithm, and generating an augmented training database including the 3D ROIs and the transformed 3D ROIs; and   training the prediction neural network model using the augmented training database.   
     
     
         19 . The method of  claim 18 , wherein the augmenting the training database further includes recalculating a position of the set of landmarks in each of the transformed 3D ROIs and associating the recalculated position of the set of landmarks with the transformed 3D ROIs. 
     
     
         20 . The method of  claim 17 , further comprising:
 resizing, for each of the set of 3D scan volumes, the 3D ROI identified for that 3D scan volume such that each 3D ROI has the same size,   the training database including the resized 3D ROI for each of the set of 3D scan volumes and the marked characteristic features associated with the set of landmarks in the resized 3D ROI.   
     
     
         21 . The method of  claim 17 , further comprising determining, using the trained prediction neural network model, a preferred placement of the medical device in an anatomical structure of a patient. 
     
     
         22 . An apparatus, comprising:
 a memory; and   a processor operatively coupled to the memory, the processor configured to:
 receive a set of three-dimensional (3D) scan volumes each including image data of an anatomical structure with a medical device implanted in the anatomical structure; 
 process each of the set of 3D scan volumes to obtain segmentation data that identifies different anatomical parts of the anatomical structure of each of the set of 3D scan volumes; 
 merge, for each of the set of 3D scan volumes, the segmentation data of that 3D scan volume with the image data of that 3D scan volume to produce combined image data for that 3D scan volume; 
 identify a 3D region of interest (ROI) in the combined image data of each of the set of 3D scan volumes that includes a subsection of the anatomical structure of that 3D scan volume with the medical device implanted therein; 
 generate a training database including (1) the 3D ROI identified for each of the set of 3D scan volumes and (2) marked characteristic features associated with a set of landmarks in the 3D ROI identified for each of the set of 3D scan volumes, the set of landmarks including an anatomical part of the anatomical structure and a portion of the medical device implanted in the anatomical structure; and 
 train, using the training database, a prediction neural network model to identify preferred positions and orientations for placement of the medical device in the anatomical structure. 
   
     
     
         23 . The apparatus of  claim 22 , wherein the processor is further configured to determine, using the trained prediction neural network model, a preferred placement of the medical device in an anatomical structure of a patient. 
     
     
         24 . The apparatus of  claim 22 , wherein the processor is further configured to:
 determine, based on a 3D scan volume of anatomical structure of the patient and segmentation data of the anatomical structure of the patient, a 3D ROI in the 3D scan volume of the anatomical structure of the patient for placement of the medical device;   process, using the trained prediction neural network model, the 3D ROI of the patient to identify the set of landmarks in the 3D ROI of the patient; and   determine a 3D position and orientation of the medical device for the preferred placement of the medical device based on the set of landmarks.   
     
     
         25 . The apparatus of  claim 24 , further comprising a display device communicably coupled to the processor, wherein the display device is configured to display the 3D position and orientation of the medical device for the preferred placement of the medical device with respect to the anatomical structure. 
     
     
         26 . The apparatus of  claim 25 , wherein the processor is further configured to:
 identify, after the medical device has been placed in the anatomical structure of the patient, an actual 3D position and orientation of the medical device; and   further train the prediction neural network model based on the actual 3D position and orientation of the medical device.   
     
     
         27 . The apparatus of  claim 22 , wherein the anatomical part of the set of landmarks includes a pedicle, and the portion of the medical device of the set of landmarks includes a tip of the medical device.