Sensor fusion using inertial and image sensors

Abstract

A method includes obtaining previous state information of a movable object, receiving inertial data from at least one inertial sensor carried by the movable object, receiving image data from at least two image sensors carried by the movable object, and estimating updated state information of the movable object based on at least one of the previous state information, the inertial data, or the image data.

What is claimed is: 
     
         1 . A method comprising:
 obtaining, with aid of one or more processors, previous state information of a movable object;   receiving, at the one or more processors, inertial data from at least one inertial sensor carried by the movable object;   receiving, at the one or more processors, image data from at least two image sensors carried by the movable object; and   estimating, with aid of the one or more processors, updated state information of the movable object based on at least one of the previous state information, the inertial data, or the image data.   
     
     
         2 . The method of  claim 1 , wherein receiving the image data includes receiving one or more first images obtained by a first image sensor of the at least two image sensors and receiving one or more second images obtained by a second image sensor of the at least two image sensors. 
     
     
         3 . The method of  claim 2 , wherein estimating the updated state information includes comparing the one or more first images and the one or more second images. 
     
     
         4 . The method of  claim 2 , wherein estimating the updated state information includes processing each of the one or more first images and the one or more second images using at least one of a feature point detection algorithm, an optical flow algorithm, or a feature matching algorithm. 
     
     
         5 . The method of  claim 1 , wherein receiving the image data includes receiving images obtained by each of the at least two image sensors over a plurality of time points during operation of the movable object. 
     
     
         6 . The method of  claim 5 , wherein the images include images of an environment around the movable object. 
     
     
         7 . The method of  claim 1 , wherein receiving the inertial data includes receiving inertial measurement data obtained by the at least one inertial sensor over a plurality of time points during operation of the movable object. 
     
     
         8 . The method of  claim 1 , wherein receiving the inertial data includes data indicative of at least one of a three-dimensional acceleration or a three-dimensional angular velocity of the movable object. 
     
     
         9 . The method of  claim 1 , wherein obtaining the previous state information includes obtaining at least one of a position, an orientation, a velocity, or an acceleration of the movable object at a previous time point during operation of the movable object. 
     
     
         10 . The method of  claim 1 , wherein obtaining the previous state information includes obtaining the previous state information using an iterative optimization algorithm. 
     
     
         11 . The method of  claim 1 , wherein obtaining the previous state information includes selecting the previous state information from a plurality of previous time points using a sliding window filter. 
     
     
         12 . The method of  claim 11 , wherein when sensor data obtained at a new time point is obtained, the new time point is added into the sliding window filter and one of the previous time points is discarded from the sliding window filter. 
     
     
         13 . The method of  claim 12 , wherein a parallax between the one of the previous time points and neighboring time points of the one of the previous time points is smaller than a threshold for a stable arithmetic solution. 
     
     
         14 . The method of  claim 1 , further comprising:
 controlling movement of the movable object based on the updated state information.   
     
     
         15 . The method of  claim 1 , wherein the movable object is an unmanned aerial vehicle. 
     
     
         16 . A system comprising:
 at least one inertial sensor and at least two image sensors carried by a movable object; and   one or more processors individually or collectively configured to:
 obtain previous state information of the movable object; 
 receive inertial data from the at least one inertial sensor; 
 receive image data from the at least two image sensors; and 
 estimate updated state information of the movable object based on at least one of the previous state information, the inertial data, or the image data. 
   
     
     
         17 . The system of  claim 16 , wherein:
 the at least two image sensors include a first image sensor and a second image sensor;   the image data includes one or more first images obtained by the first image sensor and one or more second images obtained by the second image sensor.   
     
     
         18 . The system of  claim 16 , wherein the previous state information is obtained from a plurality of previous time points using a sliding window filter. 
     
     
         19 . The system of  claim 18 , wherein when sensor data obtained at a new time point is obtained, the new time point is added into the sliding window filter and one of the previous time points is discarded from the sliding window filter. 
     
     
         20 . The system of  claim 19 , wherein a parallax between the one of the previous time points and neighboring time points of the one of the previous time points is smaller than a threshold for a stable arithmetic solution.