Learning apparatus, estimation apparatus, learning method, estimation method, and computer programs

Abstract

A learning device includes: a time-series information generation unit that obtains a first image group including a plurality of successive time-series images including a reference image and generates first time-series information based on a difference between the reference image and each of the images in the first image group other than the reference image; and a first learning unit that performs machine learning using the reference image and the first time-series information, thereby obtaining a first learning result used for estimating depth information on a target image, which is an image to be processed, and silhouette information on a subject captured in the target image based on the target image and second time-series information generated from a second image group including a plurality of successive time-series images including the target image.

The invention claimed is: 
     
       1. A learning device, comprising:
 a processor; and 
 a storage medium having computer program instructions stored thereon, where executed by the processor, perform to:
 obtain a first image group including a plurality of successive time-series images including a reference image and generates first time-series information based on a difference between the reference image and each of the images in the first image group other than the reference image; and 
 performs machine learning using the reference image and the first time-series information, thereby obtaining a first learning result that is to be used for estimating depth information on a target image, which is an image to be processed, and silhouette information on a subject captured in the target image based on the target image and second time-series information generated from a second image group including a plurality of successive time-series images including the target image. 
 
 
     
     
       2. The learning device according to  claim 1 , wherein the computer program instructions further perform to obtain the first learning result, which indicates configuration information and parameter information on a deep neural network that receives the reference image and the first time-series information and outputs the depth information on the reference image and the silhouette information on the subject captured in the reference image. 
     
     
       3. An estimation device, comprising:
 a processor; and 
 a storage medium having computer program instructions stored thereon, where executed by the processor, perform to:
 estimate the depth information on the target image and the silhouette information on the subject captured in the target image by using the first learning result obtained from the learning device according to  claim 1 , the target image and the second time-series information. 
 
 
     
     
       4. The estimation device according to  claim 3 , wherein the computer program instructions further perform to:
 perform machine learning using the silhouette information on the subject captured in the reference image, thereby obtaining a second learning result that is to be used for estimating joint information on the subject captured in the target image using the silhouette information on the subject captured in the target image; and
 estimate the joint information on the subject captured in the target image by using the silhouette information on the subject captured in the target image and the second learning result. 
 
 
     
     
       5. A learning method, comprising:
 a time-series information generation step of obtaining a first image group including a plurality of successive time-series images including a reference image and generating first time-series information based on a difference between the reference image and each of the images in the first image group other than the reference image; and 
 a learning step of performing machine learning using the reference image and the first time-series information, thereby obtaining a learning result that is to be used for estimating depth information on a target image, which is an image to be processed, and silhouette information on a subject captured in the target image based on the target image and second time-series information generated from a second image group including a plurality of successive time-series images including the target image. 
 
     
     
       6. An estimation method, comprising:
 an estimation step of estimating the depth information on the target image and the silhouette information on the subject captured in the target image by using the learning result obtained by the learning method according to  claim 5 , the target image and the second time-series information. 
 
     
     
       7. A non-transitory computer readable medium including instructions executable by one or more processor to:
 obtain a first image group including a plurality of successive time-series images including a reference image; 
 generate first time-series information based on a difference between the reference image and each of the images in the first image group other than the reference image; and 
 perform machine learning using the reference image and the first time-series information, thereby obtaining a learning result that is to be used for estimating depth information on a target image, which is an image to be processed, and silhouette information on a subject captured in the target image based on the target image and second time-series information generated from a second image group including a plurality of successive time-series images including the target image. 
 
     
     
       8. The non-transitory computer readable medium of  claim 7  further including instructions executable by one or more processor to estimate the depth information on the target image and the silhouette information on the subject captured in the target image by using the learning result obtained, the target image and the second time-series information.