Automated segmentation utilizing fully convolutional networks

Abstract

Systems and methods for automated segmentation of anatomical structures (e.g., heart). Convolutional neural networks (CNNs) may be employed to autonomously segment parts of an anatomical structure represented by image data, such as 3D MRI data. The CNN utilizes two paths, a contracting path and an expanding path. In at least some implementations, the expanding path includes fewer convolution operations than the contracting path. Systems and methods also autonomously calculate an image intensity threshold that differentiates blood from papillary and trabeculae muscles in the interior of an endocardium contour, and autonomously apply the image intensity threshold to define a contour or mask that describes the boundary of the papillary and trabeculae muscles. Systems and methods also calculate contours or masks delineating the endocardium and epicardium using the trained CNN model, and anatomically localize pathologies or functional characteristics of the myocardial muscle using the calculated contours or masks.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented machine learning method, comprising:
 training a fully convolutional neural network (CNN) model to generate a trained CNN model for segmenting an anatomical structure based, at least in part, on a plurality of images, wherein each of a subset of the plurality of images includes at least one label which identifies a region of a particular part of the anatomical structure depicted in the image, the trained CNN model comprising an expanding path that includes a plurality of convolutional layers and a plurality of upsampling layers, wherein each upsampling layer is preceded by at least one convolutional layer and comprises a fixed upsampling operation without a learned kernel and a convolution operation with a learned kernel, and the convolution operation is preceded by the fixed upsampling operation and succeeded by a concatenation of feature maps; and   storing the trained CNN model in a nontransitory processor-readable storage medium.   
     
     
         22 . The method of  claim 21 , wherein training the CNN model further comprises selecting the CNN model based, at least in part, on validation accuracy of the CNN model. 
     
     
         23 . The method of  claim 22 , further comprising performing a random search over hyperparameters associated with a set of CNN models to determine a highest validation accuracy. 
     
     
         24 . The method of  claim 23 , wherein the hyperparameters describe at least one of a model, training of the model, training data to use, or data augmentation to use during training. 
     
     
         25 . The method of  claim 21 , wherein the concatenation of feature maps is based, at least in part, on a skip connection from another path of the trained CNN model. 
     
     
         26 . The method of  claim 25 , wherein the another path is a contracting path that includes a plurality of convolutional layers and a plurality of pooling layers. 
     
     
         27 . The method of  claim 26 , wherein the number of pooling layers in the contracting path equals the number of upsampling layers in the expanding path. 
     
     
         28 . A computer-readable medium storing contents that, when executed by one or more processors, cause the one or more processors to perform actions comprising:
 training a fully convolutional neural network (CNN) model to generate a trained CNN model for segmenting an anatomical structure based, at least in part, on a plurality of images, wherein each of a subset of the plurality of images includes at least one label which identifies at least a portion of the anatomical structure depicted in the image, the trained CNN model comprising an expanding path that includes a plurality of convolutional layers and a plurality of upsampling layers, wherein each upsampling layer is preceded by at least one convolutional layer and comprises a fixed upsampling operation without a learned kernel and a convolution operation with a learned kernel, and the convolution operation is preceded by the fixed upsampling operation and succeeded by a concatenation of feature maps; and   storing the trained CNN model.   
     
     
         29 . The computer-readable medium of  claim 28 , wherein the trained CNN model further includes skip connections between layers in the expanding path and another path of the trained CNN model. 
     
     
         30 . The computer-readable medium of  claim 29 , wherein the skip connections are residual connections that add or subtract values of feature maps. 
     
     
         31 . The computer-readable medium of  claim 29 , wherein the concatenation of feature maps is based on at least one of the skip connections. 
     
     
         32 . The computer-readable medium of  claim 28 , wherein training the CNN model further comprises selecting the CNN model based, at least in part, on validation accuracy of the CNN model. 
     
     
         33 . The computer-readable medium of  claim 32 , wherein the actions further comprise performing a random search over hyperparameters associated with a set of CNN models to determine a highest validation accuracy. 
     
     
         34 . The computer-readable medium of  claim 33 , wherein the hyperparameters describe at least one of a model, training of the model, training data to use, or data augmentation to use during training. 
     
     
         35 . A system, comprising:
 at least one processor; and   memory storing contents that, when executed by the at least one processor, cause the system to:
 train a fully convolutional neural network (CNN) model to generate a trained CNN model for segmenting an anatomical structure based, at least in part, on a plurality of images, wherein each of a subset of the plurality of images includes at least one label which identifies at least a portion of the anatomical structure depicted in the image, the trained CNN model comprising an expanding path that includes a plurality of convolutional layers and a plurality of upsampling layers, wherein each upsampling layer is preceded by at least one convolutional layer and comprises a fixed upsampling operation without a learned kernel and a convolution operation with a learned kernel, and the convolution operation is preceded by the fixed upsampling operation and succeeded by a concatenation of feature maps; and 
 store the trained CNN model. 
   
     
     
         36 . The system of  claim 35 , wherein to train the CNN model, the contents further cause the system to select the CNN model based, at least in part, on validation accuracy of the CNN model. 
     
     
         37 . The system of  claim 36 , wherein the contents further cause the system to perform a random search over hyperparameters associated with a set of CNN models to determine a highest validation accuracy. 
     
     
         38 . The system of  claim 37 , wherein the hyperparameters describe at least one of a model, training of the model, training data to use, or data augmentation to use during training. 
     
     
         39 . The system of  claim 35 , wherein each upsampling layer halves the number of feature maps and doubles the spatial resolution. 
     
     
         40 . The system of  claim 39 , wherein the trained CNN model further comprises a same number of pooling layers as the upsampling layers and wherein each pooling layer doubles the number of feature maps and halves the spatial resolution.