Multi-view image analysis using neural networks

Abstract

Volumetric quantification can be performed for various parameters of an object represented in volumetric data. Multiple views of the object can be generated, and those views provided to a set of neural networks that can generate inferences in parallel. The inferences from the different networks can be used to generate pseudo-labels for the data, for comparison purposes, which enables a co-training loss to be determined for the unlabeled data. The co-training loss can then be used to update the relevant network parameters for the overall data analysis network. If supervised data is also available then the network parameters can further be updated using the supervised loss.

What is claimed is: 
     
         1 . A processor comprising:
 one or more circuits to use one or more first neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by one or more second neural networks corresponding to one or more second views of the one or more objects within one or more second images.   
     
     
         2 . The processor of  claim 1 , wherein the one or more circuits are further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         3 . The processor of  claim 2 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         4 . The processor of  claim 3 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         5 . The processor of  claim 1 , wherein the one or more circuits are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, features corresponding to the object of interest. 
     
     
         6 . The processor of  claim 5 , wherein the one or more circuits are further to generate the second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         7 . The processor of  claim 1 , wherein the segmentation information includes one or more segmentations, classifications, or regressions. 
     
     
         8 . A system comprising:
 one or more memories to store neural network parameters corresponding to one or more first neural networks; and   one or more processors to use the one or more first neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by one or more second neural networks corresponding to one or more second views of the one or more objects within one or more second images.   
     
     
         9 . The system of  claim 8 , wherein the one or more processors further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         10 . The system of  claim 9 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         11 . The system of  claim 10 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         12 . The system of  claim 8 , wherein the one or more processors are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, features corresponding to the object of interest. 
     
     
         13 . The system of  claim 12 , wherein the one or more processors are further to generate the second views in part by manipulating three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         14 . The system of  claim 8 , wherein the segmentation information includes one or more segmentations, classifications, or regressions. 
     
     
         15 . A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
 use one or first more neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by one or more second neural networks corresponding to one or more second views of the one or more objects within one or more second images.   
     
     
         16 . The non-transitory machine-readable medium of  claim 15 , wherein the one or more processors are further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         17 . The non-transitory machine-readable medium of  claim 16 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         18 . The non-transitory machine-readable medium of  claim 17 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         19 . The non-transitory machine-readable medium of  claim 15 , wherein the one or more processors are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, features corresponding to the object of interest. 
     
     
         20 . The non-transitory machine-readable medium of  claim 19 , wherein the one or more processors are further to generate second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         21 . The non-transitory machine-readable medium of  claim 15 , wherein the segmentation information includes one or more segmentations, classifications, or regressions. 
     
     
         22 . A medical imaging system comprising:
 one or more medical scanners for generating volumetric data representative of one or more features of an object of interest, the object of interest corresponding to a body of a patient to be examined; and   one or more processors to use one or more first neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by one or more second neural networks corresponding to one or more second views of the one or more objects within one or more second images.   
     
     
         23 . The medical imaging system of  claim 22 , wherein the one or more processors are further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         24 . The medical imaging system of  claim 23 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         25 . The medical imaging system of  claim 24 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         26 . The medical imaging system of  claim 22 , wherein the one or more processors are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, features corresponding to the object of interest. 
     
     
         27 . The medical imaging system of  claim 26 , wherein the one or more processors are further to generate the second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         28 . The medical imaging system of  claim 22 , wherein the one or more processors are further to select inferred segmentations corresponding to selected views or to ensembles of the second views. 
     
     
         29 . A processor comprising:
 one or more circuits to train one or more first neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by one or more second neural networks corresponding to one or more second views of the one or more objects within one or more second images.   
     
     
         30 . The processor of  claim 29 , wherein the one or more circuits are further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         31 . The processor of  claim 30 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         32 . The processor of  claim 31 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         33 . The processor of  claim 29 , wherein the one or more circuits are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, the features corresponding to the object of interest. 
     
     
         34 . The processor of  claim 33 , wherein the one or more circuits are further to generate the second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         35 . The processor of  claim 29 , wherein the segmentation information includes one or more segmentations, classifications, or regressions. 
     
     
         36 . A system comprising:
 one or more processors to train one or more first neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by one or more second neural network corresponding to one or more second views of the one or more objects within one or more second images; and   one or more memories to store neural network parameters.   
     
     
         37 . The system of  claim 36 , wherein the one or more processors are further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         38 . The system of  claim 37 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         39 . The system of  claim 38 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         40 . The system of  claim 36 , wherein the one or more processors are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, the features corresponding to the object of interest. 
     
     
         41 . The system of  claim 40 , wherein the one or more processors are further to generate the second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         42 . The system of  claim 36 , wherein the segmentation information includes one or more segmentations, classifications, or regressions. 
     
     
         43 . A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
 train one or more first neural networks to generate segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation generated by one or more second neural networks corresponding to one or more second views of one or more objects within one or more second images.   
     
     
         44 . The non-transitory machine-readable medium of  claim 43 , wherein the one or more processors are further to cause the one or more second neural networks to generate inferences for the second views in parallel. 
     
     
         45 . The non-transitory machine-readable medium of  claim 44 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         46 . The non-transitory machine-readable medium of  claim 45 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         47 . The non-transitory machine-readable medium of  claim 43 , wherein the one or more processors are further to generate the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, the features corresponding to the object of interest. 
     
     
         48 . The non-transitory machine-readable medium of  claim 47 , wherein the one or more processors are further to generate the second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features. 
     
     
         49 . The non-transitory machine-readable medium of  claim 43 , wherein the segmentation information includes one or more segmentations, classifications, or regressions. 
     
     
         50 . A method comprising:
 generating segmentation information corresponding to one or more views of one or more objects within one or more images based, at least in part, on segmentation information generated by
 one or more second neural networks corresponding to one or more second views of one or more objects within one or more second images. 
   
     
     
         51 . The method of  claim 50 , further comprising:
 causing the one or more second neural networks to generate inferences for the second views in parallel.   
     
     
         52 . The method of  claim 51 , wherein a function of loss values comprises a co-training loss function of the inferences, and wherein respective inferences are used as pseudo-labels for unlabeled data. 
     
     
         53 . The method of  claim 52 , wherein the function of loss values further comprises a supervised loss function of inferences generated for supervised data including ground truth data. 
     
     
         54 . The method of  claim 50 , further comprising:
 generating the second views from a set of volumetric data, including the one or more images, corresponding to different orientations of an object of interest represented in the volumetric data, features corresponding to the object of interest.   
     
     
         55 . The method of  claim 54 , further comprising:
 generating the second views in part by transforming three-dimensional (3D) data, of the set of volumetric data, and utilizing asymmetrical 3D kernels to encourage diversified features.   
     
     
         56 . The method of  claim 50 , wherein the segmentation information includes one or more segmentations, classifications, or regressions inferred for the second views.