Techniques to use a neural network to expand an image

Abstract

Apparatuses, systems, and techniques for texture synthesis from small input textures in images using convolutional neural networks. In at least one embodiment, one or more convolutional layers are used in conjunction with one or more transposed convolution operations to generate a large textured output image from a small input textured image while preserving global features and texture, according to various novel techniques described herein.

What is claimed is: 
     
         1 . A processor, comprising:
 a set of graphics cores to share a cache memory, wherein each of the graphics cores comprise:
 an instruction cache; 
 a cache/shared memory; 
 a texture unit; 
 a set of registers; 
 integer logic units; 
 floating point logic units to perform 16-bit floating point operations; and 
 matrix processing units (MPUs) to perform half-precision floating point and 8-bit integer operations; 
   memory coupled with the sets of graphics cores, wherein the memory includes graphics double data rate (GDDR) memory;   a memory controller; and   a PCI Express host interface,   wherein the processor is to use a neural network to generate an output image, wherein the neural network includes:
 a convolutional layer to determine feature maps; and 
 a summer to aggregate the feature maps. 
   
     
     
         2 . The processor of  claim 1 , wherein the memory controller is to provide access to a memory interface to access synchronous dynamic random access memory (SDRAM) devices. 
     
     
         3 . The processor of  claim 1 , wherein each of the graphics cores further comprise a scheduler to schedule one or more threads to be performed. 
     
     
         4 . The processor of  claim 1 , further comprising an input/output hub to couple the processor to other processor instances. 
     
     
         5 . The processor of  claim 1 , further comprising a link to enable communication between with other processor instances. 
     
     
         6 . The processor of  claim 1 , wherein the processor is to be included on a system on chip (SoC). 
     
     
         7 . The processor of  claim 1 , wherein each of the graphics cores further comprise a dispatcher to dispatch one or more threads to be performed. 
     
     
         8 . The processor of  claim 1 , further comprising floating point logic units to perform 32-bit floating point operations. 
     
     
         9 . The processor of  claim 1 , wherein the convolutional layer is to determine the feature maps based, at least in part, on an input image and the input image is smaller than the output image. 
     
     
         10 . The processor of  claim 1 , wherein the processor is to use the neural network to upsample the feature maps to generate the output image. 
     
     
         11 . A system on chip (SoC), comprising:
 a set of graphics cores to share a cache memory, wherein each of the graphics cores comprise:
 an instruction cache; 
 a cache/shared memory; 
 a texture unit; 
 a set of registers; 
 integer logic units; 
 floating point logic units to perform 16-bit floating point operations; and 
 matrix processing units (MPUs) to perform half-precision floating point and 8-bit integer operations; 
   memory coupled with the sets of graphics cores, wherein the memory includes graphics double data rate (GDDR) memory;   a memory controller; and   a PCI Express host interface,   wherein the SoC is to use a neural network to generate an output image, wherein the neural network includes:
 a convolutional layer to determine feature maps; and 
 a summer to aggregate the feature maps. 
   
     
     
         12 . The SoC of  claim 11 , wherein the memory controller is to provide access to a memory interface to access synchronous dynamic random access memory (SDRAM) devices. 
     
     
         13 . The SoC of  claim 11 , wherein each of the graphics cores further comprise a scheduler to schedule one or more threads to be performed. 
     
     
         14 . The SoC of  claim 11 , further comprising an input/output hub to couple the SoC to other processor instances. 
     
     
         15 . The SoC of  claim 11 , further comprising a link to enable communication with other processor instances. 
     
     
         16 . The SoC of  claim 11 , further comprising one or more processors optimized to perform one or more shader programs. 
     
     
         17 . The SoC of  claim 11 , wherein each of the graphics cores further comprise a dispatcher to dispatch one or more threads to be performed. 
     
     
         18 . The SoC of  claim 11 , further comprising floating point logic units to perform 32-bit floating point operations. 
     
     
         19 . The SoC of  claim 11 , wherein the convolutional layer is to determine the feature maps based, at least in part, on an input image and the input image is smaller than the output image. 
     
     
         20 . The SoC of  claim 11 , wherein the SoC is to use the neural network to upsample the feature maps to generate the output image.