Control method, control system, and smart glasses for first person view unmanned aerial vehicle flight

Abstract

A system for controlling an unmanned aerial vehicle (UAV) includes smart glasses configured to establish a first channel directly with the UAV, receive first person view (FPV) image data directly from the UAV through the first channel, and display the FPV image data, a remote control configured to establish a second channel directly with the UAV and send a first flight control instruction corresponding to the FPV image data directly to the UAV through the second channel, and establish a fourth channel directly with the mobile terminal, and a mobile terminal configured to display the FPV image data received from the UAV and forwarded by the remote control.

What is claimed is: 
     
         1 . A system for controlling an unmanned aerial vehicle (UAV) comprising:
 smart glasses configured to establish a first channel directly with the UAV, receive first person view (FPV) image data directly from the UAV through the first channel, and display the FPV image data;   a remote control configured to establish a second channel directly with the UAV and send a first flight control instruction corresponding to the FPV image data directly to the UAV through the second channel, and establish a fourth channel directly with the mobile terminal; and   a mobile terminal configured to display the FPV image data received from the UAV and forwarded by the remote control,   wherein:   the smart glasses are further configured to establish a third channel directly with the remote control for the remote control to send, through the third channel, at least one of a disabling instruction for disabling one or more menu functions of the smart glasses, a first smart glasses control instruction for adjusting a display parameter of the smart glasses, or a second smart glasses control instruction for controlling a camera setting menu for the smart glasses, to the smart glasses,
 wherein a gimbal configured on the UAV carries a first camera for collecting the FPV image data, the disabled one or more menu functions correspond to a second flight control instruction for controlling flight of the UAV or a gimble control instruction for controlling a rotation of the gimbal, and the camera setting menu is configured to set the first camera. 
   
     
     
         2 . The system according to  claim 1 , wherein the smart glasses are further configured to receive flight status data directly from the UAV through the first channel and display the flight status data. 
     
     
         3 . The system according to  claim 2 , wherein the smart glasses are further configured to display the flight status data by superimposing the flight status data on the FPV image data in an augmented reality mode. 
     
     
         4 . The system according to  claim 1 , wherein the smart glasses are further configured to:
 sense an attitude of the smart glasses;   generate the gimbal control instruction according to the attitude; and   send the gimbal control instruction directly to the UAV through the first channel.   
     
     
         5 . The system according to  claim 1 , wherein the smart glasses are further configured to:
 sense a user gesture;   generate a glasses control instruction according to the user gesture; and   perform an operation according to the glasses control instruction.   
     
     
         6 . The system according to  claim 1 , wherein:
 the smart glasses are further configured to:
 sense a user gesture; 
 generate the second flight control instruction according to the user gesture; and 
 send the second flight control instruction directly to the UAV through the first channel. 
   
     
     
         7 . The system according to  claim 6 , wherein the smart glasses include:
 a second camera configured to capture an image of a user gesture operation; and   a processor configured to:
 process the image of the user gesture operation; 
 determine coordinates of the user gesture operation; and 
 compare the coordinates of the user gesture operation with an imaging position of a virtual operation interface displayed on the smart glasses for controlling the UAV, to recognize the second flight control instruction corresponding to the user gesture operation and to perform an operation corresponding to the second flight control instruction. 
   
     
     
         8 . The system according to  claim 6 , wherein the smart glasses include:
 a second camera configured to capture a user gesture motion;   a memory for storing a plurality of user gesture motion models and a plurality of second flight control instructions corresponding to the plurality of user gesture motion models; and   a processor electrically coupled to the second camera and the memory, and configured to:
 generate a gesture motion model according to the user gesture motion; 
 compare the generated gesture motion model with the plurality of user gesture motion models stored in the memory; and 
 trigger the second flight control instruction corresponding to the generated gesture motion model in response to the generated gesture motion model being consistent with one of the plurality of user gesture motion models stored in the memory. 
   
     
     
         9 . The system according to  claim 8 , wherein the plurality of user gesture motion models stored in the memory are obtained by photographing a plurality of user gesture motions through the second camera and defining the user gesture motion models to correspond to the plurality of user gesture motions, respectively. 
     
     
         10 . The system according to  claim 1 , wherein the remote control comprises:
 buttons configured to implement the menu function of the smart glasses,
 wherein the buttons are five-dimensional buttons, and 
 wherein the menu function includes display brightness, camera settings, 
 photographing, and video recording. 
   
     
     
         11 . The system according to  claim 1 , wherein the remote control is further configured to:
 receive at least one of the FPV image data or flight status data from the UAV through the second channel.   
     
     
         12 . The system according to  claim 1 , wherein:
 the mobile terminal is configured to generate at least one of a third flight control instruction or a gimbal control instruction according to user input and send the at least one of the third flight control instruction or the gimbal control instruction to the remote control through the fourth channel; and   the remote control is further configured to send the at least one of the first flight control instruction or the gimbal control instruction to the UAV through the second channel.   
     
     
         13 . The system according to  claim 1 , wherein the smart glasses are further configured to send flight status data received from the UAV to the remote control through the third channel for forwarding to the mobile terminal through the fourth channel for display. 
     
     
         14 . The system according to  claim 13 , wherein:
 the mobile terminal is configured to generate at least one of a third flight control instruction or a gimbal control instruction according to user input and send the at least one of the third flight control instruction or the gimbal control instruction to the smart glasses through the remote control; and   the smart glasses are further configured to send the at least one of the second flight control instruction or the gimbal control instruction to the UAV through the first channel.   
     
     
         15 . The system according to  claim 1 , wherein the mobile terminal is further configured to:
 establish a wireless channel directly with the UAV;   receive the FPV image data from the UAV through the wireless channel and display the FPV image data; and   send a third flight control instruction according to user input to the UAV through the wireless channel.   
     
     
         16 . The system according to  claim 1 , wherein the smart glasses are further configured to:
 obtain a video file directly from the UAV through the first channel; and   play back the video file.   
     
     
         17 . The system according to  claim 1 , wherein the display parameter of the smart glasses includes a display backlight brightness of the smart glasses. 
     
     
         18 . A method for controlling an unmanned aerial vehicle (UAV) comprising:
 controlling smart glasses to establish a first channel directly with the UAV;   controlling the smart glasses to receive first person view (FPV) image data directly from the UAV through the first channel and to display the FPV image data;   controlling a remote control that is separate from the smart glasses to establish a second channel directly with the UAV and send a first flight control instruction corresponding to the FPV image data directly to the UAV through the second channel;   controlling a mobile terminal to display the FPV image data received from the UAV and forwarded by the smart glasses;   controlling the smart glasses to establish a third channel directly with the remote control, and controlling the remote control to send, through the third channel, at least one of a disabling instruction for disabling one or more menu functions of the smart glasses, a first smart glasses control instruction for adjusting a display parameter of the smart glasses, or a second smart glasses control instruction for controlling a camera setting menu for the smart glasses, to the smart glasses; and   controlling the remote control to establish a fourth channel directly with the mobile terminal,   wherein a gimbal configured on the UAV carries a first camera for collecting the FPV image data, the disabled one or more menu functions correspond to a second flight control instruction for controlling flight of the UAV or a gimbal control instruction for controlling a rotation of the gimbal, and the camera setting menu is configured to set the first camera.   
     
     
         19 . The method according to  claim 18 , further comprising:
 controlling the smart glasses to sense a user gesture to generate the second flight control instruction by:
 controlling a second camera disposed on the smart glasses to capture an image of a user gesture operation, processing the image of the gesture operation to determine coordinates of the user gesture operation, and comparing the coordinates of the user gesture operation with an imaging position of a virtual operation interface displayed on the smart glasses for controlling the UAV, to recognize the second flight control instruction corresponding to the user gesture operation and to perform an operation corresponding to the second flight control instruction; or 
 controlling the second camera to capture a user gesture motion, generating a gesture motion model according to the user gesture motion, comparing the gesture motion model with a plurality of pre-stored user gesture motion models in the smart glasses, and, in response to the gesture motion model being consistent with one of the plurality of pre-stored user gesture motion models, triggering the second flight control instruction corresponding to the gesture motion model; and 
   controlling the smart glasses to send the second flight control instruction directly to the UAV through the first channel.