Segmentation, landmark detection and view classification using multi-task learning

Abstract

Medical image data may be applied to a machine-learned network learned on training image data and associated image segmentations, landmarks, and view classifications to classify a view of the medical image data, detect a location of one or more landmarks in the medical image data, and segment a region in the medical image data based on the application of the medical image data to the machine-learned network. The classified view, the segmented region, or the location of the one or more landmarks may be output.

We claim: 
     
       1. A method for performing multiple diagnostic tasks on medical image data, the method comprising:
 receiving, by a processor, first medical image data; 
 applying, by the processor, the first medical image data to a machine-learned network learned on second medical image data and associated image segmentations, landmarks, and view classifications; 
 classifying, by the processor, a view of the first medical image data based on the application of the first medical image data to the machine-learned network; 
 detecting, by the processor, a location of one or more landmarks in the first medical image data based on the application of the first medical image data to the machine-learned network; 
 segmenting, by the processor, a region in the first medical image data based on the application of the first medical image data to the machine-learned network; and 
 outputting, by the processor, the classified view, the segmented region, or the location of the one or more landmarks. 
 
     
     
       2. The method of  claim 1 , further comprising:
 rescaling, by a processor, the first medical image data to match a resolution of the second medical image data. 
 
     
     
       3. The method of  claim 1 , wherein classifying the view further comprises:
 generating an anatomic label and an orientation of the first medical image data. 
 
     
     
       4. The method of  claim 1 , wherein detecting the location of the one or more landmarks is based on the view classification. 
     
     
       5. The method of  claim 1 , wherein the first medical image data is generated by an ultrasound, magnetic resonance tomography, or computed tomography imaging system. 
     
     
       6. The method of  claim 5 , wherein the second medical image data is generated by an ultrasound, magnetic resonance tomography, or computed tomography imaging system, and wherein the first medical image data is generated by a different imaging modality than at least a portion of the second medical image data. 
     
     
       7. The method of  claim 1 , wherein the processor is part of a medical imaging system. 
     
     
       8. A medical imaging system for performing multiple diagnostic tasks on medical image data, the system comprising:
 a memory storing a machine-learned network learned on second medical image data and ground truth including segmentation, landmark, and view classification for each of a plurality of second images of the second medical image data; and 
 an image processor configured to apply the medical image data to the machine-learned network and, based thereon, detect a location of one or more landmarks in the first medical image data, classify a view of the first medical image data, segment anatomy in the first medical image, or combinations thereof. 
 
     
     
       9. The system of  claim 8 , further comprising:
 an ultrasound, magnetic resonance tomography, or computed tomography medical imaging scanner configured to generate the first medical image data. 
 
     
     
       10. The system of  claim 9 , wherein the machine-learned network was trained on second medical image data having been generated by a further medical imaging scanner of a modality different from the medical imaging scanner configured to generate the first medical image data. 
     
     
       11. The method of  claim 1 , wherein the view of the first medical image data is an orientation of the first medical image data in reference to a viewing point or to a side of a body. 
     
     
       12. The method of  claim 3 , wherein the orientation of the first medical image data is referenced to a viewing point or to a side of a body. 
     
     
       13. The system of  claim 8 , wherein the view of the first medical image data is an orientation of the first medical image data in reference to a viewing point or to a side of a body.