Method and apparatus for providing driver information via audio and video metadata extraction

Abstract

A method and/or system is able to provide driver fingerprint via metadata extraction managed by a driver rating (“DR”) model trained by a machine learning center (“MLC”) coupled to a cloud based network (“CBN”). In one embodiment, a DR system includes a set of outward facing cameras, a set of inward facing cameras, and a vehicle onboard computer (“VOC”). The set of outward facing cameras mounted on a vehicle is used to collect external images representing a surrounding environment in which the vehicle operates. The set of inward facing cameras mounted in the vehicle is used to collect internal images including operator body expression representing at least operator's attention. The VOC is configured to determine the identity of operator and current operating style in response to the collected internal images, the collected external images, and historical stored data.

What is claimed is: 
     
         1 . A method configured to provide adaptive driver assistance, comprising:
 monitoring the behavior of an operator captured by one or more interior sensors and capturing surrounding information collected by one or more exterior sensors onboard a vehicle as the vehicle is in motion;   recording data related to the operator behavior and the surrounding information;   based at least in part on the recorded data, creating a digital historical profile of the operator;   detecting an abnormal driving activity from monitoring the operator behavior, the surrounding information, or both, while the vehicle is in motion; and   based on the detected abnormal driving activity and the historical profile of the operator, adaptively providing driving assistance to the operator.   
     
     
         2 . The method of  claim 1 , wherein adaptively providing driving assistance to the operator is further based on the captured surrounding information. 
     
     
         3 . The method of  claim 2 , wherein adaptively providing driving assistance to the operator includes deferring or preventing the providing of the driving assistance to the operator based on the captured surrounding information. 
     
     
         4 . The method of  claim 1 , wherein detecting the abnormal driving activity comprises:
 receiving a triggering event generated based on monitoring the behavior of the operator, capturing the surrounding information, or both;   in response to the triggering event, rewinding the recorded data related to the operator behavior and the surrounding information leading up to the receipt of the triggering event; and   generating labeled data from the recorded data, wherein the labeled data is associated with the abnormal driving activity.   
     
     
         5 . The method of  claim 4 , further comprising uploading the labeled data to a cloud-based network for training a machine-learning model. 
     
     
         6 . The method of  claim 1 , wherein the abnormal driving activity includes an infraction of driving laws or rules on the operator's part, inattention or distracted driving on the operator's part, or evasive or sudden changes in driving on the operator's part. 
     
     
         7 . The method of  claim 1 , further comprising recognizing the identity of the operator while the operator is in the vehicle. 
     
     
         8 . The method of  claim 7 , wherein the historical profile of the operator includes a skill classification of the operator, driving restrictions placed on the operator, or data related to previous assistance events involving the operator. 
     
     
         9 . The method of  claim 1 , wherein adaptively providing driver assistance includes providing driving advice or warnings through a user interface of the vehicle. 
     
     
         10 . The method of  claim 1 , wherein detecting the abnormal driving activity comprises extracting metadata from a data stream associated with monitoring the operator behavior and from a data stream associated with capturing the surrounding information. 
     
     
         11 . A system configured to provide adaptive driver assistance, comprising:
 one or more interior sensors configured to monitor the behavior of an operator of a vehicle and to provide a first data stream generated from monitoring the behavior;   one or more exterior sensors configured to capture information about the surroundings of the vehicle and to provide a second data stream generated from the captured information;   one or more processors configured to:
 analyze and record data from the first and second data streams; 
 based on the analysis, create a historical profile of the operator; 
 based on the analysis, detect abnormal driving activity on the part of the operator while the operator is driving the vehicle; and 
 based on the detected abnormal driving activity and the historical profile of the operator, adaptively provide driving assistance to the operator. 
   
     
     
         12 . The system of  claim 11 , wherein the processor is part of a vehicle-onboard computer on the vehicle that is programed with an adaptive-driver-assistance model that is configured to receive updates from a machine-learning center that is remote to the vehicle. 
     
     
         13 . The system of  claim 12 , wherein the processor is further configured to:
 return to the point in the recording of the data from the first and second data streams prior to the detection of the abnormal driving activity; and   generate labeled data from the recorded data, wherein the labeled data is associated with the abnormal driving activity.   
     
     
         14 . The system of  claim 13 , wherein the processor is further configured to cause the labeled data to be delivered to the machine-learning center to train one or more machine-learning models of the machine-learning center. 
     
     
         15 . The system of  claim 11 , wherein the processor is further configured to adaptively provide driving assistance to the operator by causing driving advice or warnings to be presented to the operator through one or more user interfaces of the vehicle. 
     
     
         16 . The system of  claim 15 , wherein the processor is further configured to adaptively provide driving assistance to the operator by causing the deferral or prevention of the presentation of driving advice or warnings to the operator. 
     
     
         17 . The system of  claim 16 , wherein the processor is further configured to adaptively provide driving assistance to the operator based on the historical profile of the operator, the captured information about the surroundings of the vehicle, or both. 
     
     
         18 . The system of  claim 11 , wherein the historical profile of the operator includes a skill classification of the operator, driving restrictions placed on the operator, or data related to previous assistance events involving the operator. 
     
     
         19 . The system of  claim 11 , the abnormal driving activity includes an infraction of driving laws or rules on the operator's part, inattention or distracted driving on the operator's part, or evasive or sudden changes in driving on the operator's part. 
     
     
         20 . A method configured to provide adaptive driver assistance, comprising:
 monitoring the behavior of an operator captured by one or more interior sensors and capturing surrounding information collected by one or more exterior sensors onboard a vehicle as the vehicle is in motion;   detecting an abnormal driving activity from the monitoring of the operator behavior, the surrounding information, or both, while the vehicle is in motion; and   based on the detected abnormal driving activity and a historical profile of the operator, adaptively providing driving assistance to the operator,   wherein the historical profile of the operator includes a skill classification of the operator, driving restrictions placed on the operator, or data related to previous assistance events involving the operator,   wherein adaptively providing driving assistance to the operator comprises adjusting a presentation of the driving assistance based one or more of the skill classification, driving restrictions, or data related to previous assistance events.