Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (mav)

Abstract

The subject matter described herein includes a modular and extensible approach to integrate noisy measurements from multiple heterogeneous sensors that yield either absolute or relative observations at different and varying time intervals, and to provide smooth and globally consistent estimates of position in real time for autonomous flight. We describe the development of the algorithms and software architecture for a new 1.9 kg MAV platform equipped with an IMU, laser scanner, stereo cameras, pressure altimeter, magnetometer, and a GPS receiver, in which the state estimation and control are performed onboard on an Intel NUC 3 rd generation i3 processor. We illustrate the robustness of our framework in large-scale, indoor-outdoor autonomous aerial navigation experiments involving traversals of over 440 meters at average speeds of 1.5 m/s with winds around 10 mph while entering and exiting buildings.

1 . A system that enables autonomous control of a vehicle in indoor and outdoor environments, the system comprising:
 a sensor fusion module for combining measurements from a plurality of sensors of different modalities to estimate a current state of the vehicle given current and previous measurements from the sensors and a previous estimated state of the vehicle, wherein the sensor fusion module is configured to maintain smoothness in the state estimates of the vehicle when: one or more sensors provide inaccurate information, when global positioning system (GPS) measurements are unavailable after a period of availability, or when GPS measurements become available after a period of unavailability; and   a trajectory generator for generating a plan for controlling a trajectory of the vehicle based on the estimated current state and a goal or a waypoint input provided by either a user or a higher level planner.   
     
     
         2 . The system of  claim 1  wherein the sensors include an inertial measurement unit (IMU), and at least one of a pressure altimeter, a magnetometer, a laser scanner, a camera, a downward facing optical sensor, and a global positioning system (GPS) receiver. 
     
     
         3 . The system of  claim 1  wherein the sensor fusion module is configured to use an Unscented Kalman Filter (UKF) to combine the measurements from the sensors of different modalities, enabling addition and removal of sensors without reconfiguration of software of the sensor fusion module. 
     
     
         4 . The system of  claim 3  wherein the sensor fusion module is configured to estimate the current state using current relative measurements and copies of augmented past states in the filter. 
     
     
         5 . The system of  claim 3  wherein the sensor fusion module is configured to judiciously remove augmented states from the filter and add new augmented states to the filter. 
     
     
         6 . The system of  claim 3  wherein the sensor fusion module is configured to fuse measurements from the sensors that arrive out of order to the filter. 
     
     
         7 . A method that enables autonomous control of a vehicle in indoor and outdoor environments, the method comprising:
 combining measurements from a plurality of sensors of different modalities to generate an estimate of a current state of the vehicle given current measurements from the sensors and a previous estimated state of the vehicle;   generating a signal for planning a trajectory of the vehicle based on the estimated current state and a goal or waypoint input by a user or a higher level planner; and   smoothing changes in state of the vehicle when: output from one or more of the sensors is inaccurate, global positioning system (GPS) measurements become available after a period of unavailability, or GPS measurements become unavailable after a period of availability.   
     
     
         8 . The method of  claim 7  wherein the sensors include at least an inertial measurement unit (IMU) and at least one of a pressure altimeter, a magnetometer, a laser scanner, a camera, and a GPS receiver. 
     
     
         9 . The method of  claim 7  wherein combining the measurements includes an Unscented Kalman Filter (UKF) to combine the measurements from the sensors of different modalities, enabling addition and removal of sensors without reconfiguration of the sensor fusion module. 
     
     
         10 . The method of  claim 9  wherein estimating the current state includes using current relative measurement and copies of augmented past states in the filter. 
     
     
         11 . The method of  claim 9  comprising removing augmented states from the filter in response to addition of a new augmented state with a binary selection matrix corresponding to that of a previous augmented state. 
     
     
         12 . The method of  claim 9  comprising fusing measurements from the sensors that arrive out of order at the filter. 
     
     
         13 . A non-transitory computer readable medium having stored thereon executable instructions that when executed by the processor of a computer controls the computer to perform steps comprising:
 combining measurements from a plurality of sensors of different modalities to generate an estimate of a current state of the vehicle given current measurements from the sensors and a previous estimated state of the vehicle;   generating a signal for planning a trajectory of the vehicle based on the estimated current state and a goal or waypoint input by a user or a higher level planner; and   smoothing changes in state of the vehicle when: output from one or more of the sensors is inaccurate, global positioning system (GPS) measurements become available after a period of unavailability, or GPS measurements become unavailable after a period of availability.   
     
     
         14 . The system of  claim 1  wherein the vehicle comprises a rotorcraft micro-aerial vehicle (MAV). 
     
     
         15 . The method of  claim 7  wherein the vehicle comprises a rotorcraft micro-aerial vehicle (MAV). 
     
     
         16 . The non-transitory computer readable medium of  claim 13  wherein the vehicle comprises a rotorcraft micro-aerial vehicle (MAV).