System and Methods for Customizing Neural Networks

Abstract

Provided herein is a system including, in some embodiments, one or more servers and one or more database servers configured to receive user-specific target information from a client application for training a neural network on a neuromorphic integrated circuit. The one or more database servers are configured to merge the user-specific target information with existing target information to form merged target information in the one or more databases. The system further includes a training set builder and a trainer. The training set builder is configured to build a training set for training a software-based version of the neural network from the merged target information. The trainer is configured to train the software-based version of the neural network with the training set to determine a set of synaptic weights for the neural network on the neuromorphic integrated circuit.

What is claimed is: 
     
         1 . A method for customizing neural networks, comprising:
 receiving, by one or more servers, user-specific target information from a client application for training a neural network on an integrated circuit;   merging, in one or more databases, the user-specific target information with existing target information to form merged target information in the one or more databases;   labeling the user-specific target information, wherein the labeling includes labeling keywords for keyword spotting; and   extracting one or more features from the labeled user-specific target information to build a training set, wherein the extracted one or more features includes time-varying frequency content in audio recordings.   
     
     
         2 . The method of  claim 1 , wherein upon a determination that a first user-specific target information includes background information which already exists in the one or more databases as an existing target information, the first user-specific target information is re-labeled as a background information. 
     
     
         3 . The method of  claim 1 , further comprising training a software-based version of the neural network on the training set to determine a set of synaptic weights for the neural network on the integrated circuit. 
     
     
         4 . The method of  claim 1 , further comprising labeling the target information in the one or more databases before merging the user-specific target information in the one or more databases, wherein the labeling includes labeling keywords for keyword spotting. 
     
     
         5 . The method of  claim 1 , wherein the training set is configured to use existing training for the existing target information. 
     
     
         6 . The method of  claim 3 , further comprising updating the synaptic weights for the software-based neural network for previously learned existing target information in view of newly learned user-specific target information, wherein the updating includes training the software-based neural network on information other than the existing target information already learned. 
     
     
         7 . The method of  claim 6 , further comprising building, with a file builder, a file of the set of synaptic weights for updating firmware of the integrated circuit including the neural network. 
     
     
         8 . The method of  claim 7 , further comprising providing the file to the client application to allow the firmware of the integrated circuit to be updated with the set of synaptic weights for the neural network. 
     
     
         9 . The method of  claim 3 , wherein the software-based neural network is configured to recognize one or more desired target information and disregard background information. 
     
     
         10 . The method of  claim 1 , further comprising
 providing a first set of desired features associated with desired target information and a second set of undesired features associated with undesired information as input to the software-based neural network; and   adjusting the synaptic weights using one or more neural network optimization algorithms, wherein the adjusting the synaptic weights are so performed that output of the neural network is positive when desired target information is recognized.   
     
     
         11 . The method of  claim 10 , wherein the one or more neural network optimization algorithms includes a gradient descent in TensorFlow. 
     
     
         12 . The method of  claim 11 , further comprising creating a plurality of synaptic weights for programming a hardware-based neural network on a platform. 
     
     
         13 . The method of  claim 12 , wherein the platform comprises at least one of: a neuromorphic integrated circuit, an analog chip, and a digital signal processor. 
     
     
         14 . The method of  claim 12 , wherein the software-based neural network and the hardware-based neural network correspond to each other. 
     
     
         15 . The method of  claim 14 , wherein the plurality of synaptic weights are mapped to physical quantities that individual circuit elements of the hardware-based neural network embody. 
     
     
         16 . A system for customizing neural networks, comprising:
 one or more database servers comprising a microprocessor configured to:
 (i) receive one or more user-specific target information, and 
 (ii) merge the user-specific target information with existing target information in one or more databases to form merged target information in the one or more databases, 
 wherein the system is configured to label the user-specific target information and extract one or more features from the labeled user-specific target information and build a training set based on the extracted information, wherein the extracted features includes time-varying frequency content in audio recordings. 
   
     
     
         17 . The system of  claim 16 , wherein the system is further configured to update synaptic weights for a software-based neural network for previously learned existing target information in view of newly learned user-specific target information. 
     
     
         18 . The system of  claim 17 , wherein the system is further configured to build a file of the synaptic weights for updating firmware of the integrated circuit including the neural network, and to provide the file to a client application, associated with the system, for updating the firmware of the integrated circuit with the set of synaptic weights for the neural network. 
     
     
         19 . The system of  claim 16 , wherein the system is further configured to re-label a first user-specific target information as background information, if the first user-specific target information includes background information which already exists in the one or more database servers as an existing target information. 
     
     
         20 . A system for customizing neural networks, comprising:
 one or more database servers comprising one or more microprocessors configured to:
 receive user-specific target information from one or more servers; 
 merge the user-specific target information with existing target information in one or more databases to form merged target information, and 
 label the user-specific target information, wherein the labeling includes labeling keywords for keyword spotting; 
   a training set builder configured to train a software-based version of the neural network from the merged target information;   a trainer configured to train a software-based version of the neural network with the training set to determine a set of synaptic weights for the neural network on the integrated circuit; and   a file builder configured to build a file of the set of synaptic weights for updating firmware of the integrated circuit including the neural network.