Method and apparatus for providing automatic mirror setting via inward facing cameras

Abstract

A method or system that is able to adjust an exterior mirror of a vehicle via an automatic mirror-setting (“AM”) model managed by a virtuous cycle containing a cloud based network (“CBN”). The system includes a set of mirrors, a set of inward facing cameras, a vehicle onboard computer (“VOC”), and AM module. In one embodiment, the mirrors, attached to a vehicle, are configured to capture at least a portion of the external environment in which the vehicle operates. The inward facing cameras, mounted in the vehicle, are configured to collect internal images, including the operator's facial features showing operator visual characteristics. The VOC, which is coupled to the CBN, is configured to determine operator vision metadata based on the internal images, operator visual characteristics, and historical stored data. The AM module is able to adaptively set a mirror to an optimal orientation so that the area of the external blind spot is minimized.

1 . A computing device, comprising:
 a memory that is configured to store computer instructions; and   one or more processors that are configured to execute the computer instructions to:
 receive at least one first image of an area external to a vehicle; 
 receive at least one second image of a driver of the vehicle; 
 analyze the at least one first image to detect movement of an object relative to the vehicle; 
 detect an abnormal event based on the movement of the object; 
 in response to the detected abnormal event, analyze the at least one second image to determine a head position of the driver within the vehicle and a gaze direction of the driver; 
 determine an orientation of at least one mirror on the vehicle based on the movement of the object and the driver's head position and gaze direction relative to the at least one mirror; 
 automatically adjust a position of the at least one mirror using the determined orientation to enable the driver to see the abnormal event via the at least one mirror; and 
 provide further automatic adjustments of the position of the at least one mirror based on tracked changes in the movement of the object. 
   
     
     
         2 . The computing device of  claim 1 , wherein the one or more processors are configured to receive the at least one second image of the driver of the vehicle by executing further computer instructions to:
 receive the at least one second image of the driver in response to the detected abnormal event.   
     
     
         3 . The computing device of  claim 1 , wherein the one or more processors are configured to receive the at least one second image of the driver of the vehicle by executing further computer instructions to:
 receive the at least one second image of the driver prior to detection of the abnormal event.   
     
     
         4 . The computing device of  claim 1 , wherein the one or more processors are configured to execute further computer instructions to:
 determine a projected location of the object based on the movement of the object;   determine the orientation of the at least one mirror on the vehicle based on the projected location of the object and the driver's head position and gaze direction relative to the at least one mirror; and   automatically adjust the position of the at least one mirror using the determined orientation to enable the driver to see the abnormal event via the at least one mirror at the projected location.   
     
     
         5 . The computing device of  claim 1 , wherein the one or more processors are configured to provide the further automatic adjustments of the position of the at least one mirror by executing further computer instructions to:
 receive at least one third image of the area external to the vehicle;   analyze the at least one third image to detect second movement of the object relative to the vehicle;   receive at least one fourth image of the driver of the vehicle;   analyze the at least one fourth image to determine a second head position of the driver within the vehicle and a second gaze direction of the driver;   determine a second orientation of at least one mirror on the vehicle based on the second movement of the object and the driver's second head position and second gaze direction relative to the at least one mirror; and   automatically adjust the position of the at least one mirror to a new position using the determined second orientation to enable the driver to maintain visual of the abnormal event via the at least one mirror.   
     
     
         6 . The computing device of  claim 1 , wherein the one or more processors are configured to execute further computer instructions to:
 provide a notice to the driver indicating the abnormal event.   
     
     
         7 . The computing device of  claim 1 , wherein the one or more processors are configured to execute further computer instructions to:
 provide a notice to the driver indicating the adjusted position of the at least one mirror in response to the abnormal event.   
     
     
         8 . A method, comprising:
 obtaining a plurality of first images of an area external to a vehicle;   obtaining a plurality of second images of a driver of the vehicle;   determining first movement of an object relative to the vehicle based on an analysis of at least a first subset of the plurality of first images;   detecting an abnormal event at a first location relative to the vehicle based on the movement of the object;   when the detected abnormal event is detected, determining a first head position of the driver based on an analysis of a first subset of the plurality of second images;   determining a first orientation of at least one mirror on the vehicle based on the first location of the abnormal event and the first head position of the driver relative to the at least one mirror;   automatically repositioning the at least one mirror using the determined first orientation to enable the driver to see the abnormal event at the first location via the at least one mirror;   determining a second location of the abnormal event relative to the vehicle based on an analysis of at least a second subset of the plurality of first images;   determining a second head position of the driver within the vehicle based on an analysis of a second subset of the plurality of second images;   determining a second orientation of the at least one mirror based on the second location of the abnormal event and the second head position of the driver relative to the at least one mirror; and   automatically repositioning the at least one mirror using the determined second orientation to enable the driver to track the abnormal event from the first location to the second location via the at least one mirror.   
     
     
         9 . The method of  claim 8 , wherein obtaining the plurality of second images of the driver of the vehicle includes:
 capturing the plurality of second images of the driver in response to the detected abnormal event.   
     
     
         10 . The method of  claim 8 , wherein obtaining the plurality of second images of the driver of the vehicle includes:
 capturing the plurality of second images of the driver prior to detection of the abnormal event.   
     
     
         11 . The method of  claim 8 , further comprising:
 determining a projected location of the object based on movement of the object from the first location to the second location;   determining a third orientation of the at least one mirror based on the projected location of the object; and   automatically repositioning the at least one mirror using the determined third orientation to enable the driver to see the abnormal event via the at least one mirror at the projected location.   
     
     
         12 . The method of  claim 8 , further comprising:
 providing a notice to the driver indicating the abnormal event.   
     
     
         13 . The method of  claim 8 , further comprising:
 providing a notice to the driver indicating the repositioning of the at least one mirror in response to the abnormal event.   
     
     
         14 . A system comprising:
 at least one mirror attached to a vehicle;   at least one first camera configured to capture a plurality of first images of an area external to the vehicle;   at least one second camera configured to capture at least one second image of a driver of the vehicle; and   a computing device, comprising:
 a memory that is configured to store computer instructions; and 
 one or more processors that are configured to execute the computer instructions to:
 obtain, via the at least one first camera, the plurality of first images of the area external to the vehicle; 
 analyze the plurality of first images to detect an abnormal event based on movement of an object relative to the vehicle; 
 in response to the detected abnormal event:
 determine a projected location of the object relative to the vehicle based on the movement of the object; 
 capture, via the at least one second camera, the at least one second image of the driver of the vehicle; 
 analyze the at least one second image to determine a gaze direction of the driver; 
 determine an orientation of the at least one mirror based on the projected location of the object and the driver's gaze direction; and 
 automatically adjust a position of the at least one mirror using the determined orientation to enable the driver to see the abnormal event via the at least one mirror when the object reaches the projected location. 
 
 
   
     
     
         15 . The system of  claim 14 , wherein the one or more processors are configured to execute further computer instructions to:
 track, based on images captured via the at least one first camera, movement of the object from the projected location to another location;   determine, based on images captured via the at least one second camera, an updated gaze direction of the driver as the object is being tracked; and   automatically adjust the position of the at least one mirror based on the tracked movement of the object and the updated driver's gaze direction to enable the driver to visually follow the abnormal event via the at least one mirror.   
     
     
         16 . The system of  claim 14 , further comprising:
 an output device, wherein the one or more processors are configured to execute further computer instructions to:   provide, via the output device, a notice to the driver indicating the abnormal event.   
     
     
         17 . The system of  claim 14 , further comprising:
 an output device, wherein the one or more processors are configured to execute further computer instructions to:   provide, via the output device, a notice to the driver indicating the adjusted position of the at least one mirror in response to the abnormal event.