Generating scalable and semantically editable font representations

Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating scalable and semantically editable font representations utilizing a machine learning approach. For example, the disclosed systems generate a font representation code from a glyph utilizing a particular neural network architecture. For example, the disclosed systems utilize a glyph appearance propagation model and perform an iterative process to generate a font representation code from an initial glyph. Additionally, using a glyph appearance propagation model, the disclosed systems automatically propagate the appearance of the initial glyph from the font representation code to generate additional glyphs corresponding to respective glyph labels. In some embodiments, the disclosed systems propagate edits or other changes in appearance of a glyph to other glyphs within a glyph set (e.g., to match the appearance of the edited glyph).

What is claimed is: 
     
       1. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to:
 determine a glyph label for a glyph image utilizing a text recognition model; 
 generate, from the glyph label and the glyph image, a font representation code for a font corresponding to the glyph image by implementing an iterative code generation process that involves utilizing a glyph appearance propagation model to generate and update font representation codes over multiple iterations by:
 iteratively generating, utilizing the glyph appearance propagation model, predicted glyph images from the glyph label for the glyph image and iteratively modified versions of a predicted font representation code; and 
 selecting, as the font representation code for the font, an iteratively modified version of the predicted font representation code corresponding to an iteratively generated predicted glyph image; and 
 
 generate, utilizing the glyph appearance propagation model, a glyph set from the font representation code and a set of glyph labels. 
 
     
     
       2. The non-transitory computer readable medium of  claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate, utilizing a first neural network within the glyph appearance propagation model, parameters for a second neural network within the glyph appearance propagation model. 
     
     
       3. The non-transitory computer readable medium of  claim 2 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the parameters for the second neural network utilizing the first neural network by:
 extracting a latent vector from the glyph label and the font representation code utilizing an encoder neural network within the first neural network; and 
 generating weights and biases for the second neural network from the latent vector utilizing a plurality of decoder neural networks within the first neural network. 
 
     
     
       4. The non-transitory computer readable medium of  claim 3 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the glyph set by predicting values indicating glyph surfaces and background areas for a set of coordinate locations utilizing the second neural network according to the weights and biases. 
     
     
       5. The non-transitory computer readable medium of  claim 1 , wherein the iterative code generation process comprises:
 iteratively generating predicted glyph images from respective predicted font representation codes and the glyph label utilizing the glyph appearance propagation model; 
 comparing the predicted glyph images to the glyph image at respective iterations; and 
 selecting, from the comparison and as the font representation code for the font corresponding to the glyph image, a predicted font representation code corresponding to a predicted glyph that satisfies a similarity metric in relation to the glyph image. 
 
     
     
       6. The non-transitory computer readable medium of  claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the glyph set by utilizing the glyph appearance propagation model to generate a new glyph image for each glyph label from the set of glyph labels according to the font representation code. 
     
     
       7. The non-transitory computer readable medium of  claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
 Receive an indication from a client device to generate a modified glyph image by applying a modification to the glyph image; and 
 automatically propagate the modification to other glyph images within the glyph set by: 
 generating an updated font representation code for an updated font corresponding to the modified glyph image; and 
 generating updated glyph images for the glyph set from the updated font representation code and the set of glyph labels. 
 
     
     
       8. The non-transitory computer readable medium of  claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the font representation code by utilizing the glyph appearance propagation model to generate a vector with a size corresponding to a plurality of anchor fonts and comprising indications of one or more of the plurality of anchor fonts contributing to a makeup of the font representation code. 
     
     
       9. A system comprising:
 one or more memory devices comprising a glyph image and a glyph appearance propagation model that is previously trained; and 
 one or more processors that are configured to cause the system to: 
 generate a font representation code for a font corresponding to the glyph image by implementing an iterative code generation process that involves utilizing the glyph appearance propagation model to generate and update font representation codes over multiple iterations by: 
 iteratively generating, utilizing the glyph appearance propagation model, predicted glyph images from a glyph label for the glyph image and iteratively modified versions of a predicted font representation code; 
 comparing the glyph image and the iteratively generated predicted glyph images; and 
 selecting, as the font representation code for the font and from the comparison, an iteratively modified version of the predicted font representation code corresponding to a predicted glyph image that satisfies a similarity metric. 
 
     
     
       10. The system of  claim 9 , wherein the one or more processors are further configured to cause the system to iteratively generate the predicted glyph images by utilizing the glyph appearance propagation model for a number of iterations to generate, for each iteration of the number of iterations, a respective predicted glyph from a respective version of the predicted font representation code and the glyph label. 
     
     
       11. The system of  claim 9 , wherein:
 comparing the glyph image and the iteratively generated predicted glyph images comprises determining, for each of the iteratively generated predicted glyph images, a loss between the glyph image and the iteratively generated predicted glyph images; and 
 selecting the iteratively modified version of the predicted font representation code comprises selecting, from the comparison, an iteratively modified version of the predicted font representation code corresponding to a predicted glyph image that satisfies a threshold loss. 
 
     
     
       12. The system of  claim 9 , wherein the one or more processors are further configured to cause the system to generate the font representation code by utilizing the glyph appearance propagation model to generate a hybrid font representation code representing an interpolation between anchor fonts. 
     
     
       13. The system of  claim 9 , wherein the one or more processors are further configured to cause the system to:
 generate, utilizing the glyph appearance propagation model, a glyph set from the font representation code and a set of glyph labels; 
 receive an indication from a client device to resize one or more glyph images of the glyph set to a larger scale; and 
 resize, without degrading appearance, the one or more glyph images to the larger scale according to parameters of the glyph appearance propagation model learned via adaptive sampling along boundaries of sample glyph images. 
 
     
     
       14. The system of  claim 9 , wherein the one or more processors are further configured to cause the system to:
 receive an indication from a client device of a modified appearance to the glyph image; and 
 automatically propagate the modified appearance to other glyph images within a common glyph set utilizing the glyph appearance propagation model. 
 
     
     
       15. The system of  claim 9 , wherein the one or more processors are further configured to cause the system to:
 receive a partial glyph image depicting an incomplete representation of a glyph; 
 generate, from the partial glyph image utilizing the glyph appearance propagation model, a different font representation code for a different font corresponding to the partial glyph image; and 
 generate, utilizing the glyph appearance propagation model, a completed glyph image from the different font representation code and a glyph label for the partial glyph image. 
 
     
     
       16. The system of  claim 9 , wherein the one or more processors are further configured to cause the system to utilize the glyph appearance propagation model comprising an encoder neural network, a plurality of decoder neural networks, and a SIREN network. 
     
     
       17. A computer-implemented method comprising:
 determining a glyph label for a glyph image utilizing a text recognition model; 
 generating, from the glyph label and the glyph image, a font representation code for a font corresponding to the glyph image by implementing an iterative code generation process that involves utilizing a glyph appearance propagation model to generate and update representation codes over multiple iterations, by:
 iteratively generating, utilizing the glyph appearance propagation model, predicted glyph images from the glyph label for the glyph image and iteratively modified versions of a predicted font representation code; and 
 selecting, as the font representation code for the font, an iteratively modified version of the predicted font representation code corresponding to an iteratively generated predicted glyph image; and 
 
 generating, utilizing the glyph appearance model, a glyph set from the font representation code and a set of glyph labels. 
 
     
     
       18. The computer-implemented method of  claim 17 , further comprising:
 receiving an indication from a client device of a modification to the glyph image; and 
 automatically propagating the modification to other glyph images within the glyph set. 
 
     
     
       19. The computer-implemented method of  claim 18 , wherein:
 receiving the indication of the modification comprises receiving an indication of one or more modified pixels of the glyph image; and 
 automatically propagating the modification comprises automatically modifying the other glyph images within the glyph set to match an appearance resulting from the one or more modified pixels of the glyph image. 
 
     
     
       20. The computer-implemented method of  claim 17 , further comprising:
 receiving an indication from a client device to resize the glyph image to a larger scale; and 
 resizing the glyph image according to the indication to the larger scale without degrading an appearance of the glyph image.