Smart windowing to reduce power consumption of a head-mounted camera used for iPPG

Abstract

Disclosed herein is utilization of windowing for efficient capturing of imaging photoplethysmogram signals (iPPG signals) with head-mounted cameras (e.g., cameras mounted to frames of smartglasses). In order to save power involved in obtaining iPPG signals, in one embodiment, a head-mounted camera with an image sensor that supports changing of its region of interest (ROI) is utilized to capture images of a region comprising skin on a user's head. A computer calculates quality scores for iPPG signals extracted from windows in the images, and selects a proper subset of the iPPG signals whose quality scores reach a threshold. The computer then reads from the camera at least one ROI that covers one or more of the windows from which the proper subset of the iPPG signals is extracted. Optionally, the at least one ROI read from the camera covers below 75% of the skin region's area.

We claim: 
     
         1 . A system configured to utilize windowing for efficient capturing of imaging photoplethysmogram signals (iPPG signals), comprising:
 an inward-facing head-mounted camera configured to capture images of a region comprising skin on a user's head (skin region) utilizing an image sensor that supports changing of its region of interest (ROI); and   a computer configured to:   calculate quality scores for iPPG signals extracted from windows in the images;   select a proper subset of the iPPG signals whose quality scores reach a threshold; and   read from the camera at least one ROI that covers one or more of the windows from which the proper subset of the iPPG signals is extracted; wherein the at least one ROI read from the camera covers below 75% of the skin region's area.   
     
     
         2 . The system of  claim 1 , wherein the quality scores for the iPPG signals are proportional to a ratio AC/DC, wherein the quality scores for the iPPG signals are proportional to a ratio AC/DC, where the AC component represents absorption of pulsatile arterial blood, and the DC component represents overall light absorption of tissue, venous blood, and non-pulsatile arterial blood. 
     
     
         3 . The system of  claim 1 , wherein the quality scores for the iPPG signals are calculated using a machine learning-based approach that utilizes at least one of the following signal quality metrics as feature values: correlation of the iPPG signals with an iPPG beat template, correlation of the iPPG signals with an iPPG beat template after linearly stretching or compressing to the length of the iPPG beat template, correlation of a resampled dynamic time warping version the iPPG signals with an iPPG beat template, percentage of the iPPG signals that are not clipped, and signal-to-noise ratios of the iPPG signals. 
     
     
         4 . The system of  claim 1 , wherein the quality scores for the iPPG signals are calculated based on a ratio of power of the iPPG signals around the pulse rate to power of noise in a passband of a bandpass filter used in the calculation of the iPPG signals. 
     
     
         5 . The system of  claim 1 , wherein the at least one ROI read from the camera covers below 10% of the skin region's area. 
     
     
         6 . The system of  claim 1 , wherein the computer is further configured to read from the camera the at least one ROI at an average frame rate higher than a maximal frame rate at which full-resolution images can be read from the camera. 
     
     
         7 . The system of  claim 1 , wherein the image sensor further supports changing its binning value, and the computer is further configured to: apply at least two different binning values to at least one of the windows, calculate at least two quality scores for iPPG signals extracted from the at least one of the windows when the at least two different binning values were applied, respectively, select a binning value with a corresponding quality score that is maximal, and read from the camera at least one of the at least one ROI according to the binning value. 
     
     
         8 . The system of  claim 7 , wherein using binning with the selected binning value reduces at least in half the time it takes the computer to read the camera compared to reading the at least one ROI in full resolution. 
     
     
         9 . The system of  claim 1 , wherein the at least one ROI comprises multiple ROIs, the image sensor supports setting multiple ROIs, and the multiple ROIs are captured simultaneously by the camera. 
     
     
         10 . The system of  claim 1 , wherein the at least one ROI comprises multiple ROIs, and the multiple ROIs are captured serially by the camera. 
     
     
         11 . The system of  claim 1 , wherein the windows are selected to cover an area expected to undergo a detectable change in hemoglobin concentration due to a certain physiological response. 
     
     
         12 . The system of  claim 11 , wherein the computer is further configured to select two different proper subsets of the iPPG signals for two different physiological responses, and to utilize two different ROIs to cover two different windows from which the two different proper subsets of the iPPG signals are extracted. 
     
     
         13 . A method comprising:
 capturing images of a region comprising skin on a user's head (skin region) utilizing an inward-facing head-mounted camera comprising an image sensor that supports changing of its region of interest (ROI);   calculating quality scores for imaging photoplethysmogram signals (iPPG signals) extracted from windows in the images;   selecting a proper subset of the iPPG signals whose quality scores reach a threshold; and   reading from the camera at least one ROI that covers one or more of the windows from which the proper subset of the iPPG signals is extracted; wherein the at least one ROI read from the camera covers below 75% of the skin region's area.   
     
     
         14 . The method of  claim 13 , further comprising calculating the quality scores for the iPPG signals using a machine learning-based approach that utilizes at least one of the following signal quality metrics as feature values: correlation of the iPPG signals with an iPPG beat template, correlation of the iPPG signals with an iPPG beat template after linearly stretching or compressing to the length of the iPPG beat template, correlation of a resampled dynamic time warping version the iPPG signals with an iPPG beat template, percentage of the iPPG signals that are not clipped, and signal-to-noise ratios of the iPPG signals. 
     
     
         15 . The method of  claim 13 , further comprising reading from the camera the at least one ROI at an average frame rate higher than a maximal frame rate at which full-resolution images can be read from the camera. 
     
     
         16 . The method of  claim 13 , wherein the image sensor further supports changing its binning value, and further comprising: applying at least two different binning values to at least one of the windows, calculating at least two quality scores for iPPG signals extracted from the at least one of the windows when the at least two different binning values were applied, respectively, selecting a binning value with a corresponding quality score that is maximal, and reading from the camera at least one of the at least one ROI according to the binning value. 
     
     
         17 . The method of  claim 13 , wherein the windows are selected to cover an area expected to undergo a detectable change in hemoglobin concentration due to a certain physiological response, and further comprising: selecting two different proper subsets of the iPPG signals for two different physiological responses, and utilizing two different ROIs to cover two different windows from which the two different proper subsets of the iPPG signals are extracted. 
     
     
         18 . A non-transitory computer readable medium storing one or more computer programs configured to cause a processor-based system to execute steps comprising:
 capturing images of a region comprising skin on a user's head (skin region) utilizing an inward-facing head-mounted camera comprising an image sensor that supports changing of its region of interest (ROI);   calculating quality scores for imaging photoplethysmogram signals (iPPG signals) extracted from windows in the images;   selecting a proper subset of the iPPG signals whose quality scores reach a threshold; and   reading from the camera at least one ROI that covers one or more of the windows from which the proper subset of the iPPG signals is extracted; wherein the at least one ROI read from the camera covers below 75% of the skin region's area.   
     
     
         19 . The non-transitory computer readable medium of  claim 18 , wherein the image sensor further supports changing its binning value, and further comprising instructions configured to cause a processor-based system to execute steps comprising: applying at least two different binning values to at least one of the windows, calculating at least two quality scores for iPPG signals extracted from the at least one of the windows when the at least two different binning values were applied, respectively, selecting a binning value with a corresponding quality score that is maximal, and reading from the camera at least one of the at least one ROI according to the binning value. 
     
     
         20 . The non-transitory computer readable medium of  claim 18 , further comprising instructions configured to cause a processor-based system to read from the camera the at least one ROI at an average frame rate higher than a maximal frame rate at which full-resolution images can be read from the camera.