Self-learning distributed system with automated ground-truth generation

Abstract

In order to generate annotated ground truth data for training a machine learning model for inferring a desired scan configuration of an medical imaging system from an observed workflow scene during exam preparation, a system is provided that comprises a sensor data interface configured to access a measurement image of a patient positioned for an imaging examination. The measurement image is generated on the basis of sensor data obtained from a sensor arrangement, which has a field of view including at least part of an area, where the patient is positioned for imaging. The system further comprises a medical image data interface configured to access a medical image of the patient obtained from a medical imaging apparatus during the imaging examination. The patient is positioned in a given geometry with respect to a reference coordinate system of the medical imaging apparatus. The system further comprises an exam metadata interface configured to access exam metadata of the imaging examination. The system further comprises a processing unit, configured to determine an association between one or more features in the measurement image and one or more features extracted from the medical image and/or from the exam metadata by mapping a point in a coordinate system of the medical image to a point in a coordinate system of the measurement image. The system further comprises an output interface, configured to be coupled to a training set database for adding the measurement image comprising data that labels the one or more associated features in the measurement image to the training set database for training the machine learning model.

The invention claimed is: 
     
       1. A system for generating annotated ground truth data for training a machine learning model for inferring a desired scan configuration of a medical imaging apparatus from an observed workflow scene during exam preparation, comprising:
 a memory; and 
 one or more processors coupled with the memory and configured to: 
 access exam metadata of an imaging examination carried out with the medical imaging apparatus, wherein the exam metadata comprises information about a configuration of the medical imaging apparatus; 
 access a measurement image of a patient positioned in a given geometry with respect to a reference coordinate system of the medical imaging apparatus for the imaging examination, wherein the measurement image is generated on the basis of sensor data obtained from a sensor arrangement, which has a field of view including at least part of an area where the patient is positioned for imaging; 
 access a medical image of the patient obtained from the medical imaging apparatus during the imaging examination; 
 determine an association between one or more features in the measurement image and one or more features extracted from the medical image and/or from the exam metadata by mapping a point in a coordinate system of the medical image to a point in a coordinate system of the measurement image; and 
 add the measurement image comprising data that labels the one or more associated features in the measurement image to a training set database for training the machine learning model. 
 
     
     
       2. The system according to  claim 1 , wherein the one or more processors are further configured to extract the one or more features from the medical image using an image processing algorithm or derive the one or more features in the medical image from the exam metadata that comprises a configuration parameter of a scan volume that was planned by an operator. 
     
     
       3. The system according to  claim 1 ,
 wherein the one or more processors are further configured to perform at least one of:
 initiating the annotated ground truth data collection; 
 interrupting the annotated ground truth data collection; and 
 stopping the annotated ground truth data collection. 
 
 
     
     
       4. The system according to  claim 1 , further comprising:
 a storage for storing the training set database obtained from one or more clinical sites. 
 
     
     
       5. The system according to  claim 1 , wherein the one or more processors are further configured to process the exam metadata to obtain an exam detail of the imaging examination comprising at least one of:
 an exam target anatomy; 
 data relating to a patient setup and an exam preparation workflow comprising a patient orientation and laterality, a presence of a specific device used for the imaging examination, and/or a trajectory during an insertion of a patient support; 
 data relating to an imaging workflow comprising a scan activity, and/or a motion of a patient support; and 
 a method for temporal alignment of data from the distributed subcomponents; and 
 wherein the one or more processors are further configured to be coupled to the training set database for adding data that comprises the exam detail of the imaging examination to the training set database for training the machine learning model. 
 
     
     
       6. The system according to  claim 1 , wherein the one or more processors are further configured to process the metadata to obtain non-image patient data of the patient comprising at least one of:
 weight, BMI, height of the patient; 
 an age of the patient; 
 a gender of the patient; 
 a medical condition of the patient comprising pregnancy, allergies to some contrast agents or others, and/or a presence of implants; 
 a quantification of a fitness level of the patient; 
 a breathing rate; 
 a pulse rate; 
 a disease diagnostic associated with the patient; 
 a medication record associated with the patient; and 
 a vital parameter record associated with the patient; and wherein the one or more processors are further configured to be coupled to the training set database for adding the non-image data to the training set database for training the machine learning model. 
 
     
     
       7. The system according to  claim 1 , wherein the sensor arrangement comprises at least one of: an optical sensor, a depth sensor, a thermal sensor, a pressure sensor, an ultrasound sensor, and an array of radio frequency sensors. 
     
     
       8. The system according to  claim 1 , wherein the desired scan configuration comprises at least one of:
 a desired patient orientation and laterality; 
 a desired scan position relative to the medical imaging apparatus; 
 an acquisition parameter for the imaging examination; and 
 a list of accessories relevant to the scan configuration. 
 
     
     
       9. The system according to  claim 1 , wherein the medical imaging apparatus comprises:
 an X-ray imaging apparatus; 
 a magnetic resonance (MR) imaging apparatus; 
 a computed tomography (CT) imaging apparatus; and/or 
 a positron-emission tomography (PET) imaging apparatus. 
 
     
     
       10. The system according to  claim 1 , wherein the medical imaging apparatus is a combined therapy/diagnostic apparatus comprising:
 an MR-Linac apparatus; 
 an MR proton therapy apparatus; and/or 
 a cone beam CT apparatus. 
 
     
     
       11. A method for generating annotated ground truth data for training a machine learning model for inferring a desired scan position from an observed workflow scene during exam preparation, the method comprising:
 accessing a measurement image of a patient positioned for an imaging examination carried out with a medical imaging apparatus, wherein the patient is positioned in a given geometry with respect to a reference coordinate system of the medical imaging apparatus, and wherein the measurement image is generated on the basis of sensor data obtained from a sensor arrangement which has a field of view including at least part of an area where the patient is positioned for imaging; 
 accessing a medical image of the patient obtained from the medical imaging apparatus during the imaging examination; 
 accessing exam metadata of the imaging examination carried out with the medical imaging apparatus, wherein the exam metadata comprises information about a configuration of the medical imaging apparatus; 
 determining an association between one or more features in the measurement image and one or more features extracted from the medical image and/or from the exam metadata by mapping a point in a coordinate system of the medical image to a point in a coordinate system of the measurement image; and 
 adding the measurement image comprising data that labels the one or more associated features in the measurement image to a training set database for the machine learning model. 
 
     
     
       12. A non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed to generate annotated ground truth data for training a machine learning model for inferring a desired scan position from an observed workflow scene during exam preparation, the method comprising:
 accessing a measurement image of a patient positioned for an imaging examination carried out with a medical imaging apparatus, wherein the patient is positioned in a given geometry with respect to a reference coordinate system of the medical imaging apparatus, and wherein the measurement image is generated on the basis of sensor data obtained from a sensor arrangement which has a field of view including at least part of an area where the patient is positioned for imaging; 
 accessing a medical image of the patient obtained from the medical imaging apparatus during the imaging examination; 
 accessing exam metadata of the imaging examination carried out with the medical imaging apparatus, wherein the exam metadata comprises information about a configuration of the medical imaging apparatus; 
 determining an association between one or more features in the measurement image and one or more features extracted from the medical image and/or from the exam metadata by mapping a point in a coordinate system of the medical image to a point in a coordinate system of the measurement image; and 
 adding the measurement image comprising data that labels the one or more associated features in the measurement image to a training set database for the machine learning model.