Systems and methods for deep model translation generation

Abstract

Embodiments of the present invention relate to systems and methods for improving the training of machine learning systems to recognize certain objects within a given image by supplementing an existing sparse set of real-world training images with a comparatively dense set of realistic training images. Embodiments may create such a dense set of realistic training images by training a machine learning translator with a convolutional autoencoder to translate a dense set of synthetic images of an object into more realistic training images. Embodiments may also create a dense set of realistic training images by training a generative adversarial network (“GAN”) to create realistic training images from a combination of the existing sparse set of real-world training images and either Gaussian noise, translated images, or synthetic images. The created dense set of realistic training images may then be used to more effectively train a machine learning object recognizer to recognize a target object in a newly presented digital image.

The invention claimed is: 
     
         1 . A method for improving the training of a computer-based object recognizer to recognize a target object within an image selected from a set of real-world images, the method comprising:
 training a translator to produce a plurality of translated images of the target object, said translator comprising a first computer-based machine learning system;   training a generative adversarial network (GAN) to produce a plurality of generated images of the target object, said GAN comprising a second computer-based machine learning system;   training the computer-based object recognizer to recognize the target object within a newly presented digital image by providing the object recognizer a collection of training images, said collection of training images assembled from:
 (1) a set of real-world images of the target object, 
 (2) the plurality of translated images of the target object, and 
 (3) the plurality of generated images of the target object. 
   
     
     
         2 . The method of  claim 1  further comprising:
 creating a set of synthetic images of the target object; and 
 wherein translator training is performed using the synthetic images. 
 
     
     
         3 . The method of  claim 2  further comprising providing the translator with a plurality of pairings, where each pairing is obtained by identifying one of the set of real-world images that corresponds with one of the set of synthetic images. 
     
     
         4 . The method of  claim 1  wherein the translator comprises a first computer-based machine learning system having a convolutional autoencoder. 
     
     
         5 . The method of  claim 1  further comprising:
 creating a set of synthetic images of the target object; 
 wherein the translator comprises a first computer-based machine learning system having a convolutional autoencoder; and 
 where translator training includes providing the translator with a plurality of pairings, where each pairing is obtained by identifying one of the set of real-world images that corresponds with one of the set of synthetic images, and where the convolutional autoencoder learns from the pairings how to produce the set of translated images of the target object. 
 
     
     
         6 . The method of  claim 1  wherein translator training includes providing the translator with a set of images generated from the second computer-based machine learning system. 
     
     
         7 . The method of  claim 1  wherein the translator training provides the translator with a plurality of second parings, where each second pairing is obtained by identifying one of the set of real-world images that corresponds with one of the set images generated from the second computer-based machine learning system. 
     
     
         8 . The method of  claim 1  wherein the translator has a convolutional autoencoder; and
 wherein translator training includes providing the translator with a plurality of second parings, where each second pairing is obtained by identifying one of the set of real-world images that corresponds with one of the set images generated from the second computer-based machine learning system, and where the convolutional autoencoder additionally learns from the second pairings how to produce the set of translated images of the target object. 
 
     
     
         9 . The method of  claim 4  wherein the convolutional autoencoder includes a loss function that is invoked iteratively during the translator training. 
     
     
         10 . The method of  claim 1  where the object recognizer includes a loss function that is invoked iteratively during the object recognizer training. 
     
     
         11 . The method of  claim 4  wherein the convolutional autoencoder includes a first loss function that is invoked iteratively during the translator training;
 wherein the object recognizer includes a second loss function that is invoked iteratively during the object recognizer training; and 
 wherein the translator training is synchronized with the object recognizer training, where loss values from the second loss function are incorporated into calculations made by the first loss function. 
 
     
     
         12 . The method of  claim 1  wherein the generated images are produced by providing the GAN with Gaussian noise. 
     
     
         13 . The method of  claim 1  wherein the GAN uses a discriminative neural network instantiated with the set of real-world images. 
     
     
         14 . The method of  claim 1  wherein the GAN uses a discriminative neural network instantiated with the set of real-world images, where each image in the set of real-world images is labeled according to the target object. 
     
     
         15 . The method of  claim 1  wherein the GAN uses a generative neural network instantiated with Gaussian noise. 
     
     
         16 . The method of  claim 1  wherein the GAN uses a discriminative neural network and a generative neural network, and wherein GAN training includes instantiating the generative neural network with Gaussian noise and instantiating the discriminative neural network with the set of real-world images, where each image in the set of real-world images is labeled according to the target object. 
     
     
         17 . The method of  claim 13  further comprising:
 creating a set of synthetic images of the target object; 
 wherein the GAN training includes instantiating the discriminative neural network with the set of synthetic images. 
 
     
     
         18 . The method of  claim 13  further comprising:
 wherein the GAN training includes instantiating the discriminative neural network with the set of translated images. 
 
     
     
         19 . The method of  claim 1  where the object recognizer includes a loss function that is invoked iteratively during the object recognizer training.