System and method for camera-based stress determination

Abstract

A system and method for camera-based stress determination. The method includes: determining a plurality of regions-of-interest (ROIs) of a body part; determining a set of bitplanes in a captured image sequence for each ROI that represent HC changes using a trained machine learning model, the machine learning model trained with a hemoglobin concentration (HC) changes training set, the HC changes training set trained using bitplanes from previously captured image sequences of other human individuals as input and received cardiovascular data as targets; determining an HC change signal for each of the ROIs based on changes in the set of determined bitplanes; for each ROI, determining intervals between heartbeats based on peaks in the HC change signal; determining heart rate variability using the intervals between heartbeats; determining a stress level using at least one determination of a standard deviation of the heart rate variability; and outputting the stress level.

The invention claimed is: 
     
       1. A method for camera-based stress determination of a human individual, the method comprising:
 receiving an image sequence capturing a body part of the human individual; 
 processing the captured image sequence, by a trained processing unit, to determine a set of bitplanes of a plurality of images in the captured image sequence that represent hemoglobin concentration (HC) changes of the subject; 
 determining intervals between heartbeats based on peaks in the set of bitplanes that represent HC changes of the subject and using the intervals between heartbeats to determine heart rate variability; 
 determining a stress level using deviations of the heart rate variability; and 
 outputting the stress level. 
 
     
     
       2. The method of  claim 1 , wherein the trained processing unit is trained using an HC changes training set with previously captured image sequences of other human individuals as input and hemodynamic changes measured by an electrocardiograph as targets. 
     
     
       3. The method of  claim 1 , wherein the bitplanes are in a red channel, green channel, and blue channel of each image of the image sequence. 
     
     
       4. The method of  claim 1 , wherein the trained processing unit comprises implementation of a Long Short Term Memory (LSTM) neural network. 
     
     
       5. The method of  claim 4 , wherein the output of the LSTM neural network comprises a matrix of bitplane composition weights as the determined set of bitplanes. 
     
     
       6. The method of  claim 1 , wherein the body part is the individual's face. 
     
     
       7. The method of  claim 1 , wherein determining intervals between heartbeats comprises:
 applying fast Fourier transform (FFT) and band pass filtering to determine a principle frequency component; 
 using the principle frequency component, reconstructing peaks of each heartbeat; and 
 determining intervals between the reconstructed peaks. 
 
     
     
       8. The method of  claim 1 , wherein determining heart rate variability comprises generating a Poincare plot of the heartbeat intervals. 
     
     
       9. The method of  claim 8 , wherein determining the stress level comprises:
 determining a first standard deviation of points of heart rate variability in a direction perpendicular to a line of identity of the Poincare plot; 
 determining a first standard deviation of points of heart rate variability in a direction that is along the line of identity; 
 determining a measure of stress as a correlation to the second standard divided by the first standard deviation. 
 
     
     
       10. The method of  claim 9 , wherein determining the stress level further comprises performing a Fisher z-transformation. 
     
     
       11. A system for camera-based stress determination of a human individual, the system comprising at least one processing unit and a data storage, the at least one processing unit in communication with the data storage and configured to execute:
 a transdermal optical imaging (TOI) module to receive an image sequence capturing a body part of the human individual; 
 a data science module to process the captured image sequence, by a trained machine learning model, to determine a set of bitplanes of a plurality of images in the captured image sequence that represent hemoglobin concentration (HC) changes of the subject; 
 a reconstruction module to determine between heartbeats based on peaks in the set of bitplanes that represent HC changes of the subject and using the intervals between heartbeats to determine heart rate variability; 
 a stress module to determine a stress level using deviations of the heart rate variability; and 
 an output module to output the stress level. 
 
     
     
       12. The system of  claim 11 , wherein the trained machine learning model is trained using an HC changes training set with previously captured image sequences of other human individuals as input and hemodynamic changes measured by an electrocardiograph as targets. 
     
     
       13. The system of  claim 11 , wherein the bitplanes are in a red channel, green channel, and blue channel of each image of the image sequence. 
     
     
       14. The system of  claim 11 , wherein the body part is the individual's face. 
     
     
       15. The system of  claim 11 , wherein the reconstruction module determines intervals between heartbeats by:
 applying fast Fourier transform (FFT) and band pass filtering to determine a principle frequency component; 
 using the principle frequency component, reconstructing peaks of each heartbeat; and 
 determining intervals between the reconstructed peaks. 
 
     
     
       16. The system of  claim 11 , wherein the stress module determines heart rate variability by generating a Poincare plot of the heartbeat intervals. 
     
     
       17. The system of  claim 16 , wherein the stress module determines the stress level by:
 determining a first standard deviation of points of heart rate variability in a direction perpendicular to a line of identity of the Poincare plot; 
 determining a first standard deviation of points of heart rate variability in a direction that is along the line of identity; 
 determining a measure of stress as a correlation to the second standard divided by the first standard deviation. 
 
     
     
       18. The system of  claim 17 , the stress module determines the measure of stress by performing a Fisher z-transformation.