Deformable image registration using deep learning

Abstract

Systems and methods are disclosed for performing operations comprising: receiving first and second images depicting an anatomy of a subject; applying a trained machine learning model to a first data set associated with the first image and a second data set associated with the second image to estimate a biomechanically accurate DVF representing a mapping of pixels or voxels from the first image to the second image, the machine learning model trained to establish a relationship between a plurality of pairs of data sets associated with images of a patient anatomy and respective biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets; applying the estimated biomechanically accurate DVF to deform a dose from a previous treatment session.

What is claimed is: 
     
       1. A method for generating a biomechanically accurate deformation vector field (DVF) for input images, the method comprising:
 receiving first and second images depicting an anatomy of a subject; 
 applying a trained machine learning model to a first data set associated with the first image, the first data set excluding a first contour associated with the first image, and a second data set associated with the second image to estimate a biomechanically accurate DVF representing a mapping of pixels or voxels from the first image to the second image without processing the first contour of the first image, the machine learning model trained to establish a relationship between a plurality of pairs of data sets associated with images of a patient anatomy and respective biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets; and 
 applying the estimated biomechanically accurate DVF to deform a dose from a previous treatment session. 
 
     
     
       2. The method of  claim 1 , wherein the machine learning model comprises a deep convolution neural network, wherein the first contour comprises data that is overlaid on top of the first image to delineate one or more structures in the first image, and wherein the second contour comprises data that is overlaid on top of the second image to delineate one or more structures in the second image. 
     
     
       3. The method of  claim 1 , further comprising training the machine learning model by adjusting one or more parameters of the machine learning model to minimize a cost function that includes a difference between a given one of the biomechanically accurate DVF representations and predicted biomechanically accurate DVF representations generated based on respective pairs of the plurality of pairs of images. 
     
     
       4. The method of  claim 1 , wherein the first data set associated with the first image includes a fixed image and the second data set associated with the second image includes a moving image, the second data set excluding a second contour associated with the second image, the biomechanically accurate DVF being estimated without processing the first and second contours of the first and second images. 
     
     
       5. The method of  claim 1 , wherein the first data set comprises at least one of one or more portions of the first image of the anatomy, and wherein the second data set comprises at least one of one or more portions of the second image. 
     
     
       6. The method of  claim 5 , further comprising applying a second machine learning model to the first set of contours and the second set of contours to estimate the DVF representing a mapping of pixels or voxels from the first image to the second image. 
     
     
       7. The method of  claim 1 , further comprising:
 segmenting the first image to generate the first set of data comprising a first set of contours comprising the first contour; 
 segmenting the second image to generate the second set of data comprising a second set of contours comprising a second contour; and 
 applying the trained machine learning model to the first image, the first set of contours, the second image and the second set of contours to estimate a biomechanically accurate DVF representing a mapping of pixels or voxels from the first image to the second image. 
 
     
     
       8. The method of  claim 1 , wherein the biomechanically accurate DVF maintains biomechanical properties of tissues depicted in the first and second images. 
     
     
       9. The method of  claim 1 , wherein the first image comprises a computed tomography (CT) image, a synthetic CT image, a magnetic resonance (MR) image, a synthetic MR image, an ultrasound image, or a synthetic ultrasound image, and wherein the second images comprises an MR image. 
     
     
       10. The method of  claim 1 , further comprising:
 generating a set of training data comprising the biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets associated with the images, the generating of the set of training data comprising: 
 receiving a first pair of data sets associated with the images of the plurality of pairs of data sets associated with the images; and 
 applying a biomechanical deformable image registration (BDIR) technique to the first pair of data sets to generate a first biomechanically accurate DVF representation. 
 
     
     
       11. The method of  claim 10 , further comprising training the machine learning model by:
 obtaining the first pair of data sets and the first biomechanically accurate DVF representation; 
 applying the machine learning model to the first pair of data sets to generate a first estimated biomechanically accurate DVF representation; 
 computing a deviation between the first estimated biomechanically accurate DVF representation and the first biomechanically accurate DVF representation; and 
 adjusting one or more parameters of the machine learning model based on the computed deviation. 
 
     
     
       12. A method of training a machine leaning model to estimate a biomechanically accurate deformation vector field (DVF) representation, the method comprising:
 obtaining a first data set associated with a first pair of images of a patient anatomy, the first data set excluding contours of the first pair of images; 
 obtaining a ground-truth biomechanically accurate DVF representation for the first data set associated with the first pair of images; 
 applying a machine learning model to the first data set to generate a first estimated biomechanically accurate DVF representation without processing the contours of the first pair of images; 
 computing a deviation between the first estimated biomechanically accurate DVF representation and the ground-truth biomechanically accurate DVF representation for the first data set; and 
 adjusting one or more parameters of the machine learning model based on the computed deviation. 
 
     
     
       13. The method of  claim 12 , further comprising:
 obtaining a plurality of pairs of data sets associated with images of the patient anatomy; 
 obtaining a plurality of ground-truth biomechanically accurate DVF representations associated with respective ones of the plurality of pairs of data sets associated with the images; and 
 for each of the plurality of pairs of data sets: 
 applying the machine learning model to the respective pair of data sets to generate a respective estimated biomechanically accurate DVF representation; 
 computing a deviation between the respective estimated biomechanically accurate DVF representation and the ground-truth biomechanically accurate DVF representation associated with the respective one of the pair of data sets; and 
 adjusting one or more parameters of the machine learning model based on the computed deviation. 
 
     
     
       14. The method of  claim 12 , further comprising:
 receiving the first data set associated with the first pair of images of the patient anatomy; and 
 applying a biomechanical deformable image registration (BDIR) technique to the first data set to generate the ground-truth biomechanically accurate DVF representation for the first data set. 
 
     
     
       15. A system comprising:
 one or more processors for performing operations comprising: 
 receiving first and second data sets associated respectively with first and second images depicting an anatomy of a subject; 
 applying a trained machine learning model to a first data set associated with the first image, the first data set excluding a first contour associated with the first image, and a second data set associated with the second image to estimate a biomechanically accurate DVF representing a mapping of pixels or voxels from the first image to the second image without processing the first contour of the first image, the machine learning model trained to establish a relationship between a plurality of pairs of data sets associated with images of a patient anatomy and respective biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets; and 
 applying the estimated biomechanically accurate DVF to deform a dose from a previous treatment session. 
 
     
     
       16. The system of  claim 15 , wherein the machine learning model comprises a deep convolution neural network. 
     
     
       17. The system of  claim 15 , wherein the operations further comprise training the machine learning model by adjusting one or more parameters of the machine learning model to minimize a cost function that includes a difference between a given one of the biomechanically accurate DVF representations and predicted biomechanically accurate DVF representations generated based on respective pairs of the plurality of pairs of data sets. 
     
     
       18. The system of  claim 15 , wherein the first data set associated with the first image includes a fixed image and the second data set associated with the second image includes a moving image. 
     
     
       19. The system of  claim 15 , wherein the first data set comprises at least one of one or more portions of the first image of the anatomy, and wherein the second data set comprises at least one of one or more portions of the second image. 
     
     
       20. The system of  claim 19 , wherein the operations further comprise applying a second machine learning model to the first set of contours and the second set of contours to estimate the DVF representing a mapping of pixels or voxels from the first image to the second image. 
     
     
       21. The system of  claim 15 , wherein the operations further comprise:
 segmenting the first image to generate the first set of data comprising a first set of contours comprising the first contour; 
 segmenting the second image to generate the second set of data comprising a second set of contours comprising a second contour; and 
 applying the trained machine learning model to the first image, the first set of contours, the second image and the second set of contours to estimate a biomechanically accurate DVF representing a mapping of pixels or voxels from the first image to the second image, the machine learning model being trained to establish a relationship between a plurality of pairs of data sets associated with images of a patient anatomy with corresponding sets of contours and respective biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets and corresponding sets of contours. 
 
     
     
       22. The system of  claim 15 , wherein the operations further comprise:
 generating a set of training data comprising the biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets associated with the images, the generating of the set of training data comprising: 
 receiving a first pair of data sets of the plurality of pairs of data sets; and 
 applying a biomechanical deformable image registration (BDIR) technique to the first pair of data sets to generate a first biomechanically accurate DVF representation. 
 
     
     
       23. The system of  claim 22 , wherein the operations further comprise training the machine learning model by:
 obtaining the first pair of data sets and the first biomechanically accurate DVF representation; 
 applying the machine learning model to the first pair of data sets to generate a first estimated biomechanically accurate DVF representation; 
 computing a deviation between the first estimated biomechanically accurate DVF representation and the first biomechanically accurate DVF representation; and 
 adjusting one or more parameters of the machine learning model based on the computed deviation. 
 
     
     
       24. A non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 receiving first and second data sets associated respectively with first and second images depicting an anatomy of a subject; 
 applying a trained machine learning model to a first data set associated with the first image, the first data set excluding a first contour associated with the first image, and a second data set associated with the second image to estimate a biomechanically accurate DVF representing a mapping of pixels or voxels from the first image to the second image without processing the first contour of the first image, the machine learning model trained to establish a relationship between a plurality of pairs of data sets associated with images of a patient anatomy and respective biomechanically accurate DVF representations of pixel or voxel mapping between the plurality of pairs of data sets; and 
 applying the estimated biomechanically accurate DVF to deform a dose from a previous treatment session. 
 
     
     
       25. A non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 obtaining a first data set associated with a first pair of images of a patient anatomy, the first data set excluding contours of the first pair of images; 
 obtaining a ground-truth biomechanically accurate DVF representation for the first data set associated with the first pair of images; 
 applying a machine learning model to the first data set to generate a first estimated biomechanically accurate DVF representation without processing the contours of the first pair of images; 
 computing a deviation between the first estimated biomechanically accurate DVF representation and the ground-truth biomechanically accurate DVF representation for the first data set; and 
 adjusting one or more parameters of the machine learning model based on the computed deviation.