Neural style transfer for image varietization and recognition

Abstract

Systems and methods for image recognition are provided. A style-transfer neural network is trained for each real image to obtain a trained style-transfer neural network. The texture or style features of the real images are transferred, via the trained style-transfer neural network, to a target image to generate styled images which are used for training an image-recognition machine learning model (e.g., a neural network). In some cases, the real images are clustered and representative style images are selected from the clusters.

What is claimed is: 
     
       1. A computer-implemented method comprising:
 obtaining, by a processor, a plurality of digital images, each respective digital image of the plurality of digital images including a representation of an object to be recognized, and a texture or style feature reflecting real-world conditions under which the image of the object was taken; 
 inputting, by the processor, one or more of the digital images of the plurality of digital images to a multi-layer neural network; 
 executing, by the processor, the multi-layer neural network, to decompose the texture or style feature of each of the one or more of the digital images into respective Gram matrices; 
 clustering, by the processor, the plurality of digital images into different groups according to, distances between the respective Gram matrices such that each group includes one or more of the digital images having similar texture or style features; 
 selecting, by the processor, one or more representative style images from each respective group of images; 
 training, by the processor, a style-transfer neural network for at least one of the representative style images to obtain one or more trained style-transfer neural networks; 
 transferring, by the processor, using the one or more trained style-transfer neural networks, the respective texture or style features from each representative image to respective target images to generate styled images; and 
 training an image-recognition machine learning model using the styled images. 
 
     
     
       2. The method of  claim 1 , wherein each respective trained style-transfer neural network of the one or more trained style-transfer neural networks corresponds to a respective representative texture or style feature. 
     
     
       3. The method of  claim 1 , further comprising selecting one trained style-transfer neural network based on a statistic describing the respective corresponding group of digital images. 
     
     
       4. The method of  claim 3 , further comprising selecting, via a multiplexer, the one trained style-transfer neural network from the trained style-transfer neural networks according to a size of the respective corresponding group of digital images. 
     
     
       5. The method of  claim 3 , further comprising choosing, via a multiplexer, one trained style-transfer neural network from the trained style-transfer neural networks according to a predetermined probability distribution of the respective corresponding group of digital images. 
     
     
       6. The method of  claim 1 , wherein the multi-layer neural network includes a Visual Geometry Group (VGG) network. 
     
     
       7. The method of  claim 1 , wherein selecting the representative style image comprises selecting, by the processor, the representative style image based on a determination that each representative style image is at a cluster center of each respective group. 
     
     
       8. The method of  claim 1 , further comprising reducing, via a manifold learning method, a dimension of each of the groups into respective 2D clusters. 
     
     
       9. The method of  claim 8 , further comprising visualizing, by the processor, the 2D clusters of images. 
     
     
       10. The method of  claim 1 , wherein the object to be recognized includes one or more of a letter, a number, a sign, a symbol, or a character. 
     
     
       11. The method of  claim 1 , further comprising evaluating, by the processor, the training of the image-recognition machine learning model. 
     
     
       12. An image-recognition system comprising:
 a memory configured to store a plurality of digital images, each respective digital image of the plurality of digital images including a representation of an object to be recognized, and a texture or style feature reflecting real-world conditions under which the image of the object was taken; and 
 a processor communicatively coupled to the memory, the processor being configured to:
 input one or more of the digital images of the plurality of digital images to a multi-layer neural network; 
 execute the multi-layer neural network, to decompose the texture or style feature of each of the one or more of the digital images into respective Gram matrices; 
 
 cluster the plurality of digital images into different groups according to, distances between the respective Gram matrices such that each group includes at least some of the digital images having similar texture or style features; 
 select one or more representative style images from each group of images; and 
 train a style-transfer neural network for at least one of the representative style images to obtain one or more trained style-transfer neural networks; 
 transfer, using the one or more trained style-transfer neural networks, the respective texture or style features from each representative image to respective target images to generate styled images; and 
 train an image-recognition machine learning model using the styled images. 
 
     
     
       13. The image-recognition system of  claim 12 , wherein each respective trained style-transfer neural network of the one or more trained style-transfer neural networks corresponds to a respective representative texture or style feature. 
     
     
       14. The image-recognition system of  claim 12 , wherein the processor is further configured to select one trained style-transfer neural network based on a statistic describing the respective corresponding group of digital images.