Dynamic Multi-Sensor and Multi-Robot Interface System

Abstract

An adaptive learning interface system for end-users for controlling one or more machines or robots to perform a given task, combining identification of gaze patterns, EEG channel's signal patterns, voice commands and/or touch commands. The output streams of these sensors are analysed by the processing unit in order to detect one or more patterns that are translated into one or more commands to the robot, to the processing unit or to other devices. A pattern learning mechanism is implemented by keeping immediate history of outputs collected from those sensors, analysing their individual behaviour and analysing time correlation between patterns recognized from each of the sensors. Prediction of patterns or combination of patterns is enabled by analysing partial history of sensors' outputs. A method for defining a common coordinate system between robots and sensors in a given environment, and therefore dynamically calibrating these sensors and devices, is used to share characteristics and positions of each object detected on the scene.

What is claimed is: 
     
         1 . A method for controlling a robot arm through gaze composed of at least one sensor acquiring images of one or more end-user's eyes, one or more processing units or controllers and one or more robotic devices;
 receiving at least one sequence of a plurality of images imaging a human performing at least one gesture by moving at least one eye;   performing an analysis of said sequence of a plurality of images to identify said at least one gesture; and identifying an association between said at least one gesture and at least one command to be executed; and   Instructing a robotic machine with said at least one command.   
     
     
         2 . The method of  claim 1 , wherein the processing unit estimates the position of a given object being observed by the end-user by analyzing the detected position of the pupil in the eye-tracker camera and the eye-tracker's camera location and pose relative to an external sensor in the environment, the current gaze direction located by detecting the pupil's center, and detecting an object in this line of view in the external sensor's 3D image. 
     
     
         3 . The method of  claim 2 , wherein said at least one external sensor is at least one depth map sensor. 
     
     
         4 . The method according to  claim 2 , wherein said visual data is complemented or replaced with audible data, touch sense data, and/or EEG data and any combinations thereof, said data produced by said human operating body. 
     
     
         5 . An apparatus associated with at least one processing unit comprising:
 at least one sensor for collecting a sequence of images of at least one human eye;   at least one processing unit configured for analyzing a sequence images from at least one of said sensor to identify gestures performed by moving eyes in certain directions, determining the center of the eye in each frame and determining at least one command to perform based on moving pattern, and generating at least one processing unit command;   at least one storage unit configured for saving said sequence of a plurality of images and detected at least one command, said at least one command of said processing unit, at least one processing algorithm and processed motion of said robotic device;   at least one digital communication means configured for communicating said at least one command to said at least one associated processing unit; and   housing configured for containing said at least one spatial sensor, said at least one processing unit, said storage unit and said communication unit.