Machine learning device, robot controller, robot system, and machine learning method for learning action pattern of human

Abstract

A machine learning device for a robot that allows a human and the robot to work cooperatively, the machine learning device including a state observation unit that observes a state variable representing a state of the robot during a period in that the human and the robot work cooperatively; a determination data obtaining unit that obtains determination data for at least one of a level of burden on the human and a working efficiency; and a learning unit that learns a training data set for setting an action of the robot, based on the state variable and the determination data.

What is claimed is: 
     
         1 . A machine learning device for a robot that allows a human and the robot to work cooperatively, the machine learning device comprising:
 a state observation unit that observes a state variable representing a state of the robot when the human and the robot work cooperatively;   a determination data obtaining unit that obtains determination data for at least one of a level of burden on the human and a working efficiency; and   a learning unit that learns a training data set for setting an action of the robot, based on the state variable and the determination data.   
     
     
         2 . The machine learning device according to  claim 1 , wherein the state variable comprises at least one of a position, an orientation, a velocity, and an acceleration of the robot. 
     
     
         3 . The machine learning device according to  claim 1 , wherein the determination data comprises at least one of a magnitude and a direction of a load sensed by the robot, a magnitude and a direction of a load sensed by an environment surrounding the robot, a level of burden on the environment surrounding the robot, and a time taken to move the robot. 
     
     
         4 . The machine learning device according to  claim 1 , wherein
 the training data set comprises an action value variable representing a value of an action of the robot set for each state of the robot and each action of the robot, and   the learning unit comprises:   a reward computation unit that sets a reward, based on the determination data and the state variable; and   a function update unit that updates the action value variable, based on the reward and the state variable.   
     
     
         5 . The machine learning device according to  claim 4 , wherein the reward computation unit sets a greater reward for a smaller absolute value of an acceleration of the robot, and a greater reward for a shorter time taken to move the robot. 
     
     
         6 . The machine learning device according to  claim 1 , wherein
 the training data set comprises a learning model for the robot set for each state of the robot and each action of the robot, and   the learning unit comprises:   an error computation unit that computes an error of the learning model, based on the determination data, the state variable, and input teacher data; and   a learning model update unit that updates the learning model, based on the error and the state variable.   
     
     
         7 . The machine learning device according to  claim 1 , further comprising:
 a human identification unit that identifies a human who works cooperatively with the robot,   wherein the training data set is created for each human, and   the learning unit learns the training data set for the identified human.   
     
     
         8 . The machine learning device according to  claim 1 , wherein the machine learning device comprises a neural network. 
     
     
         9 . The machine learning device according to  claim 1 , wherein the robot comprises one of an industrial robot, a field robot, and a service robot. 
     
     
         10 . A robot controller comprising:
 the machine learning device according to  claim 1 ; and   an action control unit that controls an action of the robot,   the machine learning device comprising a decision unit that sets an action of the robot, based on the training data set,   wherein the action control unit controls the action of the robot, based on a command from the decision unit.   
     
     
         11 . A robot system comprising:
 the robot controller according to  claim 10 ;   a robot that assists a human in work; and   an end effector attached to the robot.   
     
     
         12 . The robot system according to  claim 11 , wherein the robot comprises:
 a force detector that outputs a signal corresponding to a force from the human; and   a state detector that detects a position and an orientation of the robot,   the determination data obtaining unit obtains the determination data, based on output of the force detector, and   the state observation unit obtains the state variable, based on output of the state detector.   
     
     
         13 . The robot system according to  claim 12 , wherein the state detector comprises at least one of a motion sensor, a pressure sensor, a torque sensor for a motor, and a contact sensor. 
     
     
         14 . The robot system according to  claim 12 , further comprising:
 a plurality of robots;   a plurality of robot controllers; and   a communication line that connects the plurality of robot controllers to each other,   wherein each of the plurality of robot controllers independently learns the training data set for a robot controlled by the robot controller and sends and shares learned information via the communication line.   
     
     
         15 . A machine learning method for a robot that allows a human and the robot to work cooperatively, the machine learning method comprising the steps of:
 observing a state variable representing a state of the robot during a period in that the human and the robot work cooperatively;   obtaining determination data for at least one of a level of burden on the human and a working efficiency; and   learning a training data set for setting an action of the robot, based on the state variable and the determination data.