Wearable multi-modal bio-sensing system

Abstract

A multi-modal bio-sensing apparatus is disclosed including a first sensor module comprising a photoplethysmogram (PPG) sensor configured to produce a first output representative of a blood volume of a human user, wherein the PPG sensor is configured to remove from the first output an error signal due to movement of a user; a second sensor module comprising an electroencephalogram (EEG) sensor configured to produce a third output representative of brain neural activity of the user; a third sensor module comprising an eye-gaze camera configured to capture a gaze direction of one or more eyes of the user; and a wireless communications transceiver coupled to receive sensor data from the first sensor module, the second sensor module, or the third sensor module and configured to wirelessly transmit the received sensor data from the first sensor module, the second sensor module, or the third sensor module out of the multi-modal bio-sensing apparatus.

1 . A multi-modal bio-sensing apparatus, comprising:
 a first sensor module comprising a photoplethysmogram (PPG) sensor configured to produce a first output representative of a blood volume of a user, wherein the PPG sensor is configured to remove from the first output an error signal due to movement of the user;   a second sensor module comprising an electroencephalogram (EEG) sensor configured to produce a third output representative of brain neural activity of the user;   a third sensor module comprising an eye-gaze camera configured to capture a gaze direction of one or more eyes of the user; and   a wireless communications transceiver coupled to receive sensor data from the first sensor module, the second sensor module, or the third sensor module and configured to wirelessly transmit the received sensor data from the first sensor module, the second sensor module, or the third sensor module out of the multi-modal bio-sensing apparatus.   
     
     
         2 . The multi-modal bio-sensing apparatus of  claim 1 , wherein the error signal is determined from a second output from an accelerometer attached to the compact multi-modal bio-sensing apparatus, and wherein the error signal is removed from the first output using an adaptive filter. 
     
     
         3 . The multi-modal bio-sensing apparatus of  claim 1 , further comprising:
 one or more galvanic skin response (GSR) sensors configured to determine an impedance of the skin of the individual.   
     
     
         4 . The multi-modal bio-sensing apparatus of  claim 1 , further comprising:
 a worldview camera configured to capture a scene around the compact multi-modal bio-sensing apparatus.   
     
     
         5 . The multi-modal bio-sensing apparatus of  claim 1 , further comprising:
 a battery power source to provide power to the first sensor module, the second sensor module, the third sensor module, and the wireless communications receiver, wherein the compact multi-modal bio-sensing apparatus is mobile with freedom for the user to move about.   
     
     
         6 . The multi-modal bio-sensing apparatus of  claim 1 , further comprising:
 a headphone or speaker;   at least one processor and at least one memory containing executable instructions to cause the data to be sent to another transceiver; and   at least another memory configured to store the data prior to transmission.   
     
     
         7 . The multi-modal bio-sensing apparatus of  claim 1 , wherein the eye-gaze camera comprises an infrared camera. 
     
     
         8 . The multi-modal bio-sensing apparatus of  claim 1 , wherein the EEG sensor comprises:
 a plurality of electrode sensors, each electrode sensor structured to include an electrode tip that is electrically conductive and an electrically conductive cage formed to enclose the electrode tip to form a Faraday cage to shield the electrode tip from external electromagnetic interference; and   an EEG control module coupled to the electrode sensors to apply and receive electrical signals from the electrode sensors.   
     
     
         9 . The multi-modal bio-sensing apparatus of claim of  claim 8 , wherein the electrode tip comprises silver and epoxy, and the Faraday cage is formed by an electrically conductive tape. 
     
     
         10 . The multi-modal bio-sensing apparatus of claim of  claim 9 , wherein the electrically conductive tape comprises copper (Cu). 
     
     
         11 . The multi-modal bio-sensing apparatus of  claim 8 , wherein each electrode sensor includes an amplifier circuit coupled to the electrode tip to provide electrical signal amplification, and the wherein the amplifier circuit is enclosed by the Faraday cage. 
     
     
         12 . The multi-modal bio-sensing apparatus of  claim 4 , wherein one or more objects captured on video from the worldview camera are identified in the gaze direction by computer vision, and an associated time-stamp recorded to indicate one or more event times around which sensor data is recorded. 
     
     
         13 . The multi-modal bio-sensing apparatus of  claim 12 , wherein the computer vision is trained on one or more classes of objects. 
     
     
         14 . The multi-modal bio-sensing apparatus of  claim 1 , wherein every data point of at least the PPG sensor, the EEG sensor, and the eye gaze camera is time-stamped for data synchronization. 
     
     
         15 . A multi-modal bio-sensing method, comprising:
 sensing, by a photoplethysmogram (PPG) sensor, a blood volume of a user and generating an output representative of the blood volume;   removing, from the output, an error signal due to a movement of the user;   sensing, by an electroencephalogram (EEG) sensor, brain neural activity of the user;   determining, by an eye-gaze camera, a gaze direction of one or more eyes of the user; and   transmitting, by a wireless transceiver, one or more of data representative of the blood volume with the error signal removed, data representative of brain neural activity, or the gaze direction of the user.   
     
     
         16 . The multi-modal bio-sensing method of  claim 15 , wherein the error signal is determined from an accelerometer, and wherein the error signal is removed from the output using an adaptive filter. 
     
     
         17 . The multi-modal bio-sensing method of  claim 15 , further comprising:
 sensing, by one or more galvanic skin response (GSR) sensors, an impedance of the skin of the user.   
     
     
         18 . The multi-modal bio-sensing method of  claim 15 , further comprising:
 capturing, by a worldview camera, a scene in an area around one or more of the PPG sensor, the EEG sensor, or the eye-gaze camera.   
     
     
         19 . The multi-modal bio-sensing method of  claim 15 , further comprising:
 powering, by a battery power source, one or more of the PPG sensor, the EEG sensor, or the eye-gaze camera, and the wireless transceiver.   
     
     
         20 . The multi-modal bio-sensing method of  claim 15 , wherein the eye-gaze camera comprises an infrared camera. 
     
     
         21 . The multi-modal bio-sensing method of  claim 15 , wherein the EEG sensor comprises:
 a plurality of electrode sensors, each electrode sensor structured to include an electrode tip that is electrically conductive and an electrically conductive cage formed to enclose the electrode tip to form a Faraday cage to shield the electrode tip from external electromagnetic interference; and   an EEG control module coupled to the electrode sensors to apply and receive electrical signals from the electrode sensors.   
     
     
         22 . The multi-modal bio-sensing method of  claim 21 , wherein
 the electrode tip comprises silver and epoxy, and wherein the Faraday cage is formed by an electrically conductive tape.   
     
     
         23 . The multi-modal bio-sensing method of  claim 22 , wherein the electrically conductive tape includes copper (Cu). 
     
     
         24 . The multi-modal bio-sensing method of  claim 21 , wherein each electrode sensor includes an amplifier circuit coupled to the electrode tip to provide electrical signal amplification, and wherein the amplifier circuit is enclosed by the Faraday cage. 
     
     
         25 . The multi-modal bio-sensing method of  claim 18 , wherein one or more objects captured on video from the worldview camera are identified in the gaze direction by computer vision, and an associated time-stamp recorded to indicate one or more event times around which sensor data is recorded. 
     
     
         26 . The multi-modal bio-sensing method of  claim 18 , wherein the computer vision is trained on one or more classes of objects. 
     
     
         27 . The multi-modal bio-sensing method of  claim 15 , wherein every data point of at least the PPG sensor, the EEG sensor, and the eye gaze camera is time-stamped for data synchronization.