Processing-target image creating device, processing-target image creating method, and operation assisting system

Abstract

A processing-target image creating device creates a processing-target image based on a plurality of input images. The processing-target image creating device includes a coordinates-associating part configured to associate coordinates on each input image plane on which each input image is positioned, coordinates on a space model onto which each input image is projected, and coordinates on a processing-target image plane, and an image adjusting part configured to adjust the processing-target image by determining for each input image a re-projection angle of a re-projection line joining the coordinate and the coordinate. The image adjusting part determines for each input image slopes of the re-projection lines such that each coordinate on the processing-target image plane corresponding to an infinity line in each input image corresponds with each other on a dividing line between image parts corresponding to each input image on the processing-target image plane.

What is claimed is: 
     
       1. An image creating device comprising:
 a memory; and 
 a processor coupled to the memory, and configured to 
 create an output image based on a plurality of input images captured by a plurality of cameras installed at different heights, the output image having a shape of a partial circle; and 
 associate coordinates on a plurality of input image planes in each of which a corresponding one of the plurality of input images is positioned, coordinates on a space model onto which the plurality of input images are projected, coordinates on a re-projection plane onto which the plurality of input images projected on the space model are re-projected, and coordinates on an output image plane in which the output image is positioned, 
 wherein the plurality of input images include respective feature images representing a predetermined feature, and a slope of a re-projection line relative to a re-projection axis that is a central axis of the space model is adjusted independently for each of the plurality of input images such that the feature images of the plurality of input images are continuous in a circumferential direction of the partial circle in the output image, the re-projection line being a straight line joining first coordinates on the space model and second coordinates on the re-projection plane corresponding to the first coordinates, 
 the predetermined feature is an infinity line or a white line horizontally drawn on a wall surface around a running body of a jib crane or a turning body of a shovel, 
 when the predetermined feature is the infinity line, the slope of the re-projection line relative to the re-projection axis is adjusted independently for each of the plurality of input images such that positions corresponding to the infinity line in a plurality of image parts of the output image are continuous, the plurality of image parts corresponding to the plurality of input images, so that a sky appears same in the plurality of image parts, and 
 when the predetermined feature is the white line, the slope of the re-projection line relative to the re-projection axis is adjusted independently for each of the plurality of input images such that respective image portions of the plurality of image parts of the output image, which image portions represent the white line, are continuous, the plurality of image parts corresponding to the plurality of input images. 
 
     
     
       2. The image creating device as claimed in  claim 1 , wherein the processor is configured to
 update the slope of the re-projection line relative to the re-projection axis independently for each of the plurality of input images in response to a change in a cylindrical radius of the space model, such that a continuity of the predetermined feature is maintained through the plurality of image parts of the output image, the plurality of image parts corresponding to the plurality of input images; and 
 change allocation rates of a horizontal image area and a road surface image area while maintaining the continuity of the predetermined feature in the output image. 
 
     
     
       3. An image creating device comprising:
 a memory; and 
 a processor coupled to the memory, and configured to create an output image by synthesizing a plurality of input images captured by a plurality of cameras installed at different heights, 
 wherein a planar overhead image is created as the output image by processing the plurality of input images such that a viewpoint of the output image is different from viewpoints of the plurality of input images, the output image including a plurality of image parts corresponding to respective imaging ranges of the plurality of cameras, 
 the plurality of input images include respective feature images representing a feature extending over the plurality of input images, 
 an offset between the feature images on a dividing line between the plurality of image parts corresponding to the plurality of input images is reduced in the output image, and 
 the feature is a white line horizontally drawn on a wall surface around a running body of a jib crane or a turning body of a shovel. 
 
     
     
       4. The image creating device as claimed in  claim 3 , wherein
 the output image having a shape of a partial circle, and 
 the feature is continuous in a circumferential direction of the partial circle. 
 
     
     
       5. The image creating device as claimed in  claim 3 , wherein
 the plurality of cameras are installed at different heights, 
 coordinates of each of the plurality of input images are associated with coordinates of the output image via coordinates of a space model with the coordinates of the space model and the coordinates of the output image being associated by straight lines, and 
 inclinations of the straight lines are individually determined to be different from each other such that coordinates of the output image corresponding to an image of the feature in one of the plurality of input images coincide with coordinates of the output image corresponding to an image of the feature in another one of the plurality of input images adjacent to said one of the plurality of input images. 
 
     
     
       6. The image creating device as claimed in  claim 3 , wherein the processor is configured to
 associate coordinates on a plurality of input image planes in each of which a corresponding one of the plurality of input images is positioned, coordinates on a space model onto which the plurality of input images are projected, coordinates on a re-projection plane onto which the plurality of input images projected on the space model are re-projected, and coordinates on an output image plane in which the output image is positioned, and 
 adjust the output image by determining a slope of a re-projection line relative to a re-projection axis that is a central axis of the space model independently for each of the plurality of input images, the re-projection line being a straight line joining first coordinates on the space model and second coordinates on the re-projection plane corresponding to the first coordinates. 
 
     
     
       7. The image creating device as claimed in  claim 3 ,
 wherein the plurality of cameras are attached to the running body of the jib crane, and 
 wherein, in the output image, an image of surroundings viewed from the running body in a horizontal direction is shown around an image of a neighborhood of the running body viewed from above the running body, and a horizon is shown in an arc shape. 
 
     
     
       8. The image creating device as claimed in  claim 3 , wherein
 the plurality of cameras are attached to the turning body of the shovel, and 
 in the output image, an image of surroundings viewed from the turning body in a horizontal direction is shown around an image of a neighborhood of the turning body viewed from above the turning body, and a horizon is shown in an arc shape. 
 
     
     
       9. An image creating device comprising:
 a memory; and 
 a processor coupled to the memory, and configured to create an output image by synthesizing a plurality of input images captured by a plurality of cameras installed at different heights, 
 wherein a planar overhead image is created as the output image by processing the plurality of input images such that a viewpoint of the output image is different from viewpoints of the plurality of input images, the output image including a plurality of image parts corresponding to respective imaging ranges of the plurality of cameras, 
 the plurality of input images include respective feature images representing a feature extending over the plurality of input images, 
 an offset between the feature images on a dividing line between the plurality of image parts corresponding to the plurality of input images is reduced in the output image, 
 the feature is an infinity line, and 
 an offset between positions corresponding to the infinity line in the plurality of image parts of the output image on the dividing line is reduced, so that a sky appears same in the plurality of image parts. 
 
     
     
       10. The image creating device as claimed in  claim 9 , wherein
 the positions corresponding to the infinity line in the plurality of image parts of the output image are caused to be continuous or move based on a position of the infinity line input by an operator. 
 
     
     
       11. The image creating device as claimed in  claim 9 , wherein
 the positions corresponding to the infinity line in the plurality of image parts of the output image are caused to be continuous or move directly by an input of a position of the infinity line by an operator. 
 
     
     
       12. The image creating device as claimed in  claim 9 , wherein
 the plurality of input images are a left image, a back image, and a right image captured by a left camera, a back camera, and a right camera provided on a left, a back, and a right of a turning body of a shovel, respectively, and 
 the offset between the positions corresponding to the infinity line in the plurality of image parts of the output image on the dividing line is reduced, the plurality of image parts corresponding to the left image, the back image, and the right image. 
 
     
     
       13. The image creating device as claimed in  claim 9 , wherein
 the output image having a shape of a partial circle, and 
 the feature is continuous in a circumferential direction of the partial circle. 
 
     
     
       14. The image creating device as claimed in  claim 9 , wherein
 the plurality of cameras are installed at different heights, 
 coordinates of each of the plurality of input images are associated with coordinates of the output image via coordinates of a space model with the coordinates of the space model and the coordinates of the output image being associated by straight lines, and 
 inclinations of the straight lines are individually determined to be different from each other such that coordinates of the output image corresponding to an image of the feature in one of the plurality of input images coincide with coordinates of the output image corresponding to an image of the feature in another one of the plurality of input images adjacent to said one of the plurality of input images. 
 
     
     
       15. The image creating device as claimed in  claim 9 , wherein the processor is configured to
 associate coordinates on a plurality of input image planes in each of which a corresponding one of the plurality of input images is positioned, coordinates on a space model onto which the plurality of input images are projected, coordinates on a re-projection plane onto which the plurality of input images projected on the space model are re-projected, and coordinates on an output image plane in which the output image is positioned, and 
 adjust the output image by determining a slope of a re-projection line relative to a re-projection axis that is a central axis of the space model independently for each of the plurality of input images, the re-projection line being a straight line joining first coordinates on the space model and second coordinates on the re-projection plane corresponding to the first coordinates. 
 
     
     
       16. The image creating device as claimed in  claim 9 ,
 wherein the plurality of cameras are attached to a running body of a jib crane, and 
 wherein, in the output image, an image of surroundings viewed from the running body in a horizontal direction is shown around an image of a neighborhood of the running body viewed from above the running body, and a horizon is shown in an arc shape. 
 
     
     
       17. The image creating device as claimed in  claim 9 , wherein
 the plurality of cameras are attached to a turning body of a shovel, and 
 in the output image, an image of surroundings viewed from the turning body in a horizontal direction is shown around an image of a neighborhood of the turning body viewed from above the turning body, and a horizon is shown in an arc shape.