Automated segmentation of three dimensional bony structure images

Abstract

A computer-implemented system: at least one processor communicably coupled to at least one nontransitory processor-readable storage medium storing processor-executable instructions or data receives segmentation learning data comprising a plurality of batches of labeled anatomical image sets, each image set comprising image data representative of a series of slices of a three-dimensional bony structure, and each image set including at least one label which identifies the region of a particular part of the bony structure depicted in each image of the image set, wherein the label indicates one of a plurality of classes indicating parts of the bone anatomy; trains a segmentation CNN, that is a fully convolutional neural network model with layer skip connections, to segment semantically at least one part of the bony structure utilizing the received segmentation learning data; and stores the trained segmentation CNN in at least one nontransitory processor-readable storage medium of the machine learning system.

1 . A method, comprising:
 receiving, at a processor, a training set including sets of labeled anatomical images, each set of labeled anatomical images including a set of two-dimensional (2D) images of a three-dimensional (3D) scan volume of patient anatomy, each labeled anatomical image of each set of labeled anatomical images being associated with one or more labels each identifying a different anatomical part of a set of anatomical parts of an anatomical structure in a portion of the patient anatomy depicted in that labeled anatomical image;   processing, in iterations, the sets of labeled anatomical images using a segmentation convolutional neural network (CNN) to produce segmentation outputs associated with the set of anatomical parts for each set of labeled anatomical images;   adjusting, after each iteration, one or more parameters of the segmentation CNN based on a difference between the segmentation output produced in that iteration and the one or more labels associated with the set of labeled anatomical images processed in that iteration; and   in response to meeting a predetermined criterion, storing the segmentation CNN in a storage medium operatively coupled to the processor.   
     
     
         2 . The method of  claim 1 , wherein the anatomical structure is the spine, and the set of anatomical parts is a set of spine parts including one or more of: a vertebral body, a pedicle, a transverse process, a lamina, or a spinous process. 
     
     
         3 . The method of  claim 1 , further comprising:
 calculating, after each iteration, a value of a loss function representative of the difference between the segmentation output produced in that iteration and the one or more labels associated with the set of labeled anatomical images processed in that iteration,   the one or more parameters of the segmentation CNN being adjusted based on the value of the loss function.   
     
     
         4 . The method of  claim 1 , wherein the one or more parameters of the segmentation CNN include one or more weights of the segmentation CNN. 
     
     
         5 . The method of  claim 1 , wherein the sets of labeled anatomical images are first sets of labeled anatomical images, the method further comprising:
 validating an accuracy of the segmentation CNN by processing a validation set including a second sets of labeled anatomical images using the segmentation CNN to produce a set of segmentation outputs; and   determining one or more performance metrics of the segmentation CNN based on the set of segmentation outputs.   
     
     
         6 . The method of  claim 5 , wherein the predetermined criterion includes the one or more performance metrics indicating that the segmentation CNN has improved. 
     
     
         7 . The method of  claim 1 , wherein the predetermined criterion includes the iterations reaching a predefined number of epochs. 
     
     
         8 . The method of  claim 1 , wherein the training set is a segmentation training set, the method further comprising:
 receiving, at the processor, a denoising training set, the denoising training set including sets of low quality images paired with sets of high quality images, each high quality image having a lower noise level than the low quality image with which that high quality image is paired; and   training a denoising CNN by processing the sets of low quality images using the denoising CNN to produce denoised outputs and adjusting one or more parameters of the denoising CNN based on differences between the denoised outputs and the sets of high quality images.   
     
     
         9 . The method of  claim 8 , further comprising:
 processing the sets of labeled anatomical images using the trained denoising CNN to denoise the sets of labeled anatomical images,   the sets of labeled anatomical images being processed using the segmentation CNN after the sets of labeled anatomical images have been denoised.   
     
     
         10 . The method of  claim 1 , further comprising:
 processing, using the trained segmentation CNN, a set of input anatomical images to produce segmentation data associated with the set of anatomical parts;   
     
     
         11 . The method of  claim 10 , further comprising:
 generate, using the segmentation data, a set of output anatomical images including image data from the set of input anatomical images and information identifying one or more anatomical parts of the set of anatomical parts in the image data.   
     
     
         12 . The method of  claim 10 ,
 generating, using the segmentation data, a segmented 3D anatomical model of the anatomical structure; and   displaying a visual representation of the segmented 3D anatomical model in which each anatomical part of the set of anatomical parts is displayed using different representation parameters.   
     
     
         13 . The method of  claim 10 , wherein the segmentation data includes per-class probabilities for each pixel of each image of the set of input anatomical images, the pre-class probabilities for each pixel including a probability of that pixel belonging to a class from a set of classes, the set of classes corresponding to the set of anatomical parts. 
     
     
         14 . The method of  claim 1 , further comprising augmenting the training set by:
 transforming a subset of images from the sets of labeled anatomical images using one or more transformations including one or more of: rotation, scaling, movement, horizontal flip, or additive noise of Gaussian or Poisson distribution and Gaussian blur.   
     
     
         15 . A method, comprising:
 receiving, at a processor, one or more sets of anatomical images, each set of anatomical images including a set of two-dimensional (2D) images of a three-dimensional (3D) scan volume of patient anatomy;   processing, for each set of anatomical images at a time, that set of anatomical images using a segmentation convolutional neural network (CNN) trained to segment an anatomical structure to produce segmentation data for that set of anatomical images, the segmentation data associated with a set of anatomical parts of the anatomical structure; and   generating a segmented 3D anatomical model of the anatomical structure at least in part by combining the segmentation data produced for the sets of anatomical images, the segmentation 3D anatomical model including information identifying the set of anatomical parts.   
     
     
         16 . The method of  claim 15 , wherein the anatomical structure is the spine, and the set of anatomical parts is a set of spine parts including one or more of: a vertebral body, a pedicle, a transverse process, a lamina, or a spinous process. 
     
     
         17 . The method of  claim 15 , further comprising:
 pre-processing the sets of anatomical images based on pre-processing performed to images of a training set used to train the segmentation CNN.   each set of anatomical images being processed using the segmentation CNN after the pre-processing of that set of anatomical images.   
     
     
         18 . The method of  claim 15 , further comprising:
 displaying a visual representation of the segmented 3D anatomical model in which each anatomical part of the set of anatomical parts is displayed using different representation parameters.   
     
     
         19 . The method of  claim 18 , wherein the representation parameters include at least one of: a color, an opacity, or a decimation. 
     
     
         20 . The method of  claim 18 , wherein the visual representation is a polygonal mesh of the anatomical structure. 
     
     
         21 . The method of  claim 15 , wherein the segmentation data includes per-class probabilities for each pixel of the images of each set of anatomical images, the pre-class probabilities for each pixel including a probability of that pixel belonging to a class from a set of classes, the set of classes corresponding to the set of anatomical parts. 
     
     
         22 . The method of  claim 15 , wherein the segmentation CNN is a fully convolutional network with skip connections between layers of the fully convolutional network. 
     
     
         23 . The method of  claim 15 , wherein the segmentation CNN includes a contracting path including convolutional layers, pooling layers, and dropout layers, where each pooling or dropout layer is preceded by at least one convolutional layer. 
     
     
         24 . The method of  claim 15 , wherein the segmentation CNN includes an expanding path including convolutional layers, upsampling layers, and a concatenation of feature maps from previous layers of the segmentation CNN, where each upsampling layer is preceded by at least one convolutional layer. 
     
     
         25 . The method of  claim 15 , further comprising:
 processing the sets of anatomical images using a denoising CNN trained to denoise anatomical images,   each set of anatomical images being processed using the segmentation CNN after the denoising of that set of anatomical images.