Machine learning-based prediction of physiological parameters in remote medical information exchange

Abstract

System and method for remote medical information exchange are disclosed. The system comprises a patient-end digital device configured to capture video stream of face of the patient. The system comprises a doctor-end digital device configured to show the video stream of the face of the patient and one or more physiological inference in accordance with the video stream. The system comprises a processor configured to receive a facial image from the captured video stream. The processor can provide the facial image to a trained neural network configured to predict a sentiment of the patient based on features in the facial image. The processor is configured to receive the sentiment of the patient from the trained neural network indicating how the patient is feeling. The processor is further configured to provide the predicted sentiment to the doctor-end digital device to aid in diagnosis of the patient.

We claim as follows: 
     
         1 . A system for remote medical information exchange, comprising:
 a patient-end digital device, configured to capture video stream of face of a patient;   a doctor-end digital device, configured to show the video stream of the face of the patient, and one or more physiological inference in accordance with the video stream;   a processor, electrically coupled with a communication network, configured to:
 receive from the patient-end digital device a facial image from the captured video stream including facial and upper body features of the patient; 
 process the facial image to generate a frontal facial image; 
 provide the frontal facial image, comprising facial and upper body features, to a first trained neural network model configured to predict a sentiment of the patient based on the facial and upper body features; 
 receive the sentiment of the patient from the first trained neural network model indicating how the patient is feeling; wherein the first trained neural network model is a regression deep learning convolutional neural network model; and 
 provide the predicted sentiment of the patient as at least one of the one or more physiological inference to the doctor-end digital device. 
   
     
     
         2 . The system of  claim 1 , wherein the first trained neural network model has a plurality of input parameters, the input parameters including three color channels corresponding to one or more images. 
     
     
         3 . The system of  claim 1 , wherein the regression deep learning convolutional neural network model is a Network-in-Network neural network model. 
     
     
         4 . The system of  claim 1 , wherein the frontal facial image is provided via three color channels. 
     
     
         5 . The system of  claim 1 , further comprising a server, electrically coupled with the communication network, and wherein the first trained neural network model is stored on the server. 
     
     
         6 . The system of  claim 1 , wherein the processor is further configured to:
 receive from the patient-end digital device, demographic data and biographic data of the patient;   provide the demographic data and biographic data, to a second trained neural network model configured to predict risks for the patient;   receive the predicted risk for the patient from the second trained neural network model; and   provide the predicted risk for the patient as at least one of the one or more physiological inference to the doctor-end digital device.   
     
     
         7 . The system of  claim 6 , wherein the demographic data includes at least one of a location of the patient-end digital device, a zip code in which the patient-end digital device is positioned, a state in which the patient-end digital device is positioned, and a country in which the patient-end digital device is positioned. 
     
     
         8 . The system of  claim 6 , where the biographic data includes at least one of an age of the patient, a dietary information of the patient, and a lifestyle information of the patient. 
     
     
         9 . The system of  claim 8 , wherein the lifestyle information of the patient can include data about physical activity of the patient. 
     
     
         10 . The system of  claim 6 , wherein the predicted risk for the patient can include the risk for medical conditions including at least one of Asthma, Allergies, Flu, and Infectious diseases. 
     
     
         11 . A computer-implemented method of predicting a physiological parameter value of a patient based on a facial image of the patient, the method including:
 receiving the facial image of the patient from a captured video stream including facial and upper body features of the patient;   processing the facial image to generate a frontal facial image;   providing the frontal facial image, comprising facial and upper body features, to a first trained neural network model configured to predict a sentiment of the patient based on the facial and upper body features;   receiving the sentiment of the patient from the first neural network model indicating how the patient is feeling; wherein the first trained neural network model is a regression deep learning convolutional neural network model; and   providing the predicted physiological parameter value including the sentiment of the patient to aid in real time diagnosis of the patient.   
     
     
         12 . The method of  claim 11 , wherein the first trained neural network model has a plurality of input parameters, the input parameters including three color channels corresponding to one or more images. 
     
     
         13 . The method of  claim 11 , wherein the regression deep learning convolutional neural network model is a Network-in-Network neural network model. 
     
     
         14 . The method of  claim 11 , wherein the frontal facial image is provided via three color channels. 
     
     
         15 . The method of  claim 11 , further including:
 receiving demographic data and biographic data of the patient;   providing the demographic data and biographic data, to a second trained neural network model configured to predict risks for the patient;   receiving the predicted risks for the patient from the second trained neural network model; and   providing the predicted physiological parameter value including the risks for the patient to aid in real time diagnosis of the patient.   
     
     
         16 . The method of  claim 15 , wherein the demographic data includes at least one of a location, a zip code, a state, and a country. 
     
     
         17 . The method of  claim 15 , where the biographic data includes at least one of an age of the patient, a dietary information of the patient, and a lifestyle information of the patient. 
     
     
         18 . The method of  claim 17 , wherein the lifestyle information of the patient can include data about physical activity of the patient. 
     
     
         19 . The method of  claim 15 , wherein the predicted risk for the patient can include the risk for medical conditions including at least one of Asthma, Allergies, Flu, and Infectious diseases. 
     
     
         20 . A non-transitory computer readable medium including a set of instructions that are executable by one or more processors of a computer to cause the computer to perform a method for predicting a physiological parameter value of a patient based on a facial image of the patient, the method including:
 receiving the facial image of the patient from a captured video stream including facial and upper body features of the patient;   processing the facial image to generate a frontal facial image;   providing the frontal facial image, comprising facial and upper body features, to a trained neural network model configured to predict a sentiment of the patient based on the facial and upper body features;   receiving the sentiment of the patient from the trained neural network model indicating how the patient is feeling; wherein the trained neural network model is a regression deep learning convolutional neural network model; and   providing the predicted physiological parameter value including the sentiment of the patient to aid in real time diagnosis of the patient.