Controller, control method using controller, and control system

Abstract

A controller for controlling an operation of a machine or a robot for handling an object by includes: a sensor signal acquisition unit that acquires a signal from a sensor that outputs three-dimensional point cloud data and a corresponding camera image of an imaging subject; a region acquisition unit that acquires a region having a height lower than a predetermined value; a non-object region setting unit that sets, as a non-object region, a region of the camera image corresponding to the acquired region; a complementary color setting unit that sets a color of the non-object region to a color complementary to a color of the object; an object position recognition unit that recognizes a position of the object relative to the imaging subject from the camera image; and an operation control unit that controls the operation of the machine or the robot using at least the recognized position.

What is claimed is: 
     
       1. A controller for controlling an operation of a machine or a robot for handling an object by recognizing a position of the object, the controller comprising:
 a memory configured to store a program; and 
 a processor configured to execute the program and control the controller to: 
 acquire a signal from a sensor that outputs three-dimensional point cloud data and that also outputs a corresponding camera image of an imaging subject on which the object may be placed; 
 determine from the three-dimensional point cloud data of the imaging subject that an object is present on the imaging subject by determining that there is a region having a height higher than a predetermined value and acquire a region having a height lower than the predetermined value from the three-dimensional point cloud data of the imaging subject; 
 set, as a non-object region, a region of the camera image corresponding to the region acquired; 
 set a color of the non-object region to a color complementary to a color of the object; 
 recognize a position of the object relative to the imaging subject from the camera image in which the color of the region set as the non-object region has been set to the complementary color; 
 recognize a height of the object from the three-dimensional point cloud data, and set a level at which the robot is to grip the object; and 
 control the operation of the machine or the robot using at least the position and the height recognized. 
 
     
     
       2. The controller according to  claim 1 , wherein the object is an object being machined, and the imaging subject is a pallet or a conveyor having thereon the object being machined. 
     
     
       3. The controller according to  claim 1 , wherein the processor is further configured to execute the program and control the controller to estimate a height of a portion of the imaging subject for which the acquisition of three-dimensional point cloud data is unsuccessful, based on three-dimensional point cloud data of a surrounding portion. 
     
     
       4. The controller according to  claim 1 , further comprising a second memory configured to store therein the color of the object, wherein
 the processor is further configured to execute the program and control the controller to determine the color complementary to the color of the object based on the color of the object stored in the second memory. 
 
     
     
       5. A control system comprising:
 the controller according to  claim 1 ; 
 a machine or a robot whose operation for handling an object is controlled by the controller; and 
 a sensor configured to output, to the controller, three-dimensional point cloud data and a corresponding camera image of an imaging subject including the object. 
 
     
     
       6. A control method executed by a controller including a memory configured to store a program and a processor for controlling an operation of a machine or a robot for handling an object by recognizing a position of the object, the processor executing the program to perform operations of the control method comprising:
 acquiring a signal from a sensor that outputs three-dimensional point cloud data and that also outputs a corresponding camera image of an imaging subject on which the object may be placed; 
 determining from the three-dimensional point cloud data of the imaging subject that an object is present on the imaging subject by determining that there is a region having a height higher than a predetermined value and acquiring a region having a height lower than the predetermined value from the three-dimensional point cloud data of the imaging subject; 
 setting, as a non-object region, a region of the camera image corresponding to the region acquired; 
 setting a color of the non-object region to a color complementary to a color of the object; 
 recognizing a position of the object from the camera image in which the color of the region set as the non-object region has been set to the complementary color; 
 recognizing a height of the object from the three-dimensional point cloud data, and set a level at which the robot is to grip the object; and 
 controlling the operation of the machine or the robot using at least the position and the height recognized.