Apparatus and methods for hierarchical training of robots

Abstract

An apparatus and methods for training and/or operating a robotic device to perform a composite task comprising a plurality of subtasks. Subtasks may be arranged in a hierarchy. Individual tasks of the hierarchy may be operated by a respective learning controller. Individual learning controllers may interface to appropriate components of feature extractor configured to detect features in sensory input. Individual learning controllers may be trained to produce activation output based on occurrence of one or more relevant features and using training input. Output of a higher level controller may be provided as activation indication to one or more lower level controllers. Inactive activation indication may be utilized to deactivate one or more components thereby improving operational efficiency. Output of a given feature extractor may be shared between two or more learning controllers.

What is claimed is: 
     
         1 . A non-transitory machine-readable storage medium having instructions embodied thereon, the instructions configured for execution by one or more processors, which when executed cause the one or more processors to:
 determine an occurrence of a feature in a sensory input provided by a sensor component of the robotic apparatus;   obtain a training input for a learning prediction component, the learning prediction component configured to produce a control output for the robotic apparatus;   at a first time instance:
 evaluate a state of an activation indication; and 
 based on an active state of the activation indication:
 cause the learning prediction component to produce a first version of the control output consistently with the feature and the training input; and 
 
 adjust a parameter of the learning prediction component based on a discrepancy measure between the first version of the control output and the training input, the parameter adjustment configured to enable the learning prediction component to produce a second version of the control output subsequent to the adjustment, the second version of the control output characterized by a smaller discrepancy measure relative to the training input; 
   at a second time instance subsequent to the first time instance:
 evaluate the state of an activation indication; and 
 based on the active state of the activation indication:
 cause the learning prediction component to produce the second version of the control output in accordance with the occurrence of the feature; 
 
 wherein the second version of the control output is configured to cause the robotic apparatus to perform an action consistent with the occurrence of the feature; and 
 the instruction execution is configured to operate a hierarchical control process comprising a higher level component and the learning prediction component forming the lower level component with respect to the higher level component; and 
 the activation indication is produced by the higher level component based on the determination of the occurrence of the feature. 
   
     
     
         2 . A method of reducing energy use/increasing duration of autonomous operation by a robotic device, the robotic device comprising a first and a second actuator, a sensor component, a feature detection apparatus, and a output prediction apparatus comprising a first and a second predictor component, the method comprising:
 obtaining via the sensor component, one or more sensor data related to an environment of the robotic device;   determining, by the feature detection apparatus, an occurrence of a given feature in the one or more sensor data, the determining of the given feature occurrence characterized by a plurality of computation operations;   providing information related to the given feature to the first predictor component;   evaluating a state of an activation indication; and   based on the activation indication being in a first task state, operating the first predictor component to produce a first output based on the information, the first output configured to cause the robot to execute the first task comprising activation of the first actuator;   wherein:   based on the activation indication being in a second task state, operating the second predictor component to produce a second output based on the information, the second output configured to cause the robot to execute the second task comprising activation of the second actuator; and   the operation of the second predictor component is characterized by an absence of additional computational operations by the feature detection component to determine the given feature occurrence in excess of the plurality of computation operations;   where the absence of additional computational operations reduces energy use.   
     
     
         3 . The method of  claim 2 , wherein:
 the output prediction apparatus comprises a plurality of prediction components configured in a hierarchy comprising an upper and a lower level;   the lower level of the hierarchy comprises the first and the second predictor components;   the upper level of the hierarchy comprises a third predictor component of the plurality of predictor components; and   the third predictor component is configured to produce the activation indication based on analysis of the sensor data.   
     
     
         4 . The method of  claim 3 , wherein: the activation indication comprises an output of a winner takes all process and the activation indication is configured to enable operation of one of the first or the second predictor components. 
     
     
         5 . The method of  claim 2 , wherein operation of the determination, by the feature detection apparatus, of the occurrence of the given feature in the one or more sensor data comprises:
 analyzing the one or more sensor data to determine a first plurality of input features of a first type and a second plurality of input features of a second type;   determining a subset of features by randomly selecting a portion of the first input features and at least one feature from the second input features;   comparing individual features of the subset of features to corresponding features of a plurality of training feature sets, individual ones of the plurality of training feature sets comprising a number of training features, the number being equal to or greater than the quantity of features within the subset of features;   based on the comparison, determining a similarity measure for a given training set of the plurality of training feature sets, the similarity measure characterizing a similarity between the individual features of the subset and the corresponding features of the plurality of training feature sets; and   responsive to the similarity measure breaching a threshold, selecting one or more training sets from the plurality of training sets.   
     
     
         6 . The method of  claim 5 , further comprising:
 determining one or more potential control outputs, individual ones of the one or more potential control outputs being associated with a corresponding training set of the plurality of training feature sets; and   determining the first output based on a transformation obtained from the one or more potential control outputs;   wherein:
 individual ones of the plurality of training feature sets comprise features of the first type and at least one feature of the second type; 
 individual ones of the plurality of training feature sets are obtained during training operation of the robotic device, the training operation being performed responsive to receiving a training signal from the robotic device; and 
 individual ones of the one or more potential control signals being determined based on the training signal and the features of the given training set. 
   
     
     
         7 . The method of  claim 6 , wherein the similarity measure is determined based on a difference between one or more first values of the features of the subset and one or more second values of the features of the given training feature set. 
     
     
         8 . The method of  claim 6 , wherein the similarity measure is determined based on a distance metric between individual features of the subset of features and corresponding features of the given training feature set. 
     
     
         9 . The method of  claim 8 , wherein selecting one or more training feature sets comprises selecting N training feature sets associated with a lowest percentile of the distance metric, N being greater than two. 
     
     
         10 . The method of  claim 8 , wherein the transformation comprises a statistical operation performed on individual ones of the one or more potential control signals associated with the selected N training sets. 
     
     
         11 . The method of  claim 10 , wherein the statistical operation is selected from the group including mean and percentile. 
     
     
         12 . A robotic apparatus, comprising:
 an energy source characterized by an energy capacity;   a first actuator;   a sensor component configured to obtain sensor data related to an environment of the robotic apparatus;   a feature detection component configured to determine an occurrence of a feature in the sensor data, the determination of the given feature occurrence being based on the feature detection component performing a plurality of computation operations; and   an output prediction component, comprising a first predictor, and a second predictor, the output prediction component configured to:
 operate the first predictor to produce a task activation output based on information related to the given feature occurrence; 
 based on the task activation corresponding to a first task, operate the second predictor to produce a control output based on the information related to the given feature occurrence; and 
 provide the output to the actuator thereby causing/enabling the robotic apparatus to execute the first task; 
   wherein: the operation of the first predictor and the second predictor are based on a single instance of the feature detection component performing the plurality of computation operations, the single instance configured to reduce energy use associated with the first task execution.   
     
     
         13 . The apparatus of  claim 12 , wherein: the output predictor component comprises a plurality of predictors configured in a hierarchy comprising an upper level and a lower level configured such that an output of a predictor of the upper level comprises an activation indication for a predictor of the lower level. 
     
     
         14 . The apparatus of  claim 13 , wherein:
 the upper hierarchy level comprises the first predictor; and   the lower hierarchy level comprises the second predictor.   
     
     
         15 . The apparatus of  claim 14 , further comprising:
 a second actuator;   wherein:
 the output prediction component comprises a third predictor, the third predictor corresponding to the lower hierarchy level, the output prediction component further configured to: 
 based on the task activation corresponding to a second task, operate the third predictor to produce another control output based on the information related to the occurrence of the feature; and 
 provide the output to the second actuator thereby causing/enabling the robot to execute the second first task. 
   
     
     
         16 . The apparatus of  claim 15 , further comprising:
 a mobile platform comprising a wheel and a manipulator, the wheel being coupled to the first actuator, the manipulator being coupled to the second actuator;   the first task comprises activation of the first actuator;   the second task comprises activation of the second actuator; and   execution of the first and the second tasks comprises the single instance of the feature detection component performing the plurality of computation operations.   
     
     
         17 . The apparatus of  claim 15 , wherein the activation indication is configured to enable operation of one and only one of the second or the third predictors based on the detection of the occurrence of the feature. 
     
     
         18 . The apparatus of  claim 12 , wherein the robotic apparatus comprises an autonomous motorized platform comprising at least three wheels. 
     
     
         19 . The apparatus of  claim 18 , wherein the robotic apparatus comprises a manipulator comprising first and second segments and at least one motorized joint, the motorized joint configured to modify an angle formed between the first and the second segments. 
     
     
         20 . The apparatus of  claim 18 , wherein:
 the feature comprises a representation of a target object;   the first task comprises a target approach operation; and   the second task comprises a target collision avoidance operation.