Optimized artificial neural networks

Abstract

Neural network architectures are represented by symbol strings. An initial population of networks is trained and evaluated. The strings representing the fittest networks are modified according to a genetic algorithm and the process is repeated until an optimized network is produced.

What is claimed is:  
     
         1 . A method of optimizing the structure of an artificial neural network, comprising: 
 defining a neural network having an initial architecture comprised of a plurality of neurons;    defining a symbol string representing the architecture of the neural network;    providing a set of neural network inputs including a set for training and a set for evaluation;    training the neural network using the training set of inputs;    evaluating the trained neural network using the evaluation set of inputs;    modifying the symbol string representation of the neural network architecture according to a genetic algorithm;    successively training and evaluating the neural networks represented by the modified symbol strings and modifying the symbol strings representing improved neural networks.    
     
     
         2 . A method according to  claim 1 , further comprising the step of 
 selecting the fittest of the evaluated trained network according to a defined criterion; and    carrying out the step of modifying according to a genetic algorithm on the selected fittest networks.    
     
     
         3 . A method according to  claim 2 , wherein the step of training the neural network is carried out by supervised learning.  
     
     
         4 . A method according to  claim 1 , wherein the step of training the neural network is carried out by supervised learning.  
     
     
         5 . A method according to  claim 2 , wherein the symbol string representation of the neural network architecture represents the number of layers of hidden neurons, and the number of neurons with each hidden layer of the network.  
     
     
         6 . A method according to  claim 5 , wherein the network is trained by back propagation, and the symbol string representation of the neural network architecture further represents the back propagation parameters of learning rate, momentum and dispersion of initial link weights.  
     
     
         7 . An artificial neural network optimized according to  claim 1 .  
     
     
         8 . An artificial neural network optimized according to  claim 2 .  
     
     
         9 . An artificial neural network optimized according to  claim 3 .  
     
     
         10 . An artificial neural network optimized according to  claim 4 .  
     
     
         11 . An artificial neural network optimized according to  claim 5 .  
     
     
         12 . An artificial neural network optimized according to  claim 6 .  
     
     
         13 . A method of optimizing the structure of an artificial neural network, comprising: 
 defining a neural network having an initial architecture comprised of a plurality of input neurons, output neurons and hidden neurons, and a plurality of signal transmission paths for applying output signals from said input neurons to said hidden neurons and for applying output signals from said hidden neurons to said output neurons;    defining a symbol string representing the architecture of the neural network;    providing a set of neural net input-output pairs including a set for training and a set for evaluation;    training the neural network by supervised learning using the training set of input-output pairs;    evaluating the trained neural network using the evaluation set of input-output pairs;    modifying the symbol string representation of the neural network architecture according to a generic algorithm;    successively training and evaluating the neural networks represented by the modified symbol strings and modifying the symbol strings representing improved neural networks.    
     
     
         14 . A method according to  claim 13 , wherein the step of training by supervised learning is carried out by a back propagation algorithm.  
     
     
         15 . A method according to  claim 14 , wherein the symbol string representation of the neural network architecture represents the number of intermediate neurons.  
     
     
         16 . A method according to  claim 15 , wherein the symbol string representation of the neural network architecture further represents the back propagation parameters of learning rate, momentum and dispersion of initial link weights.  
     
     
         17 . An artificial neural network optimized according to  claim 13 .  
     
     
         18 . An artificial neural network optimized according to  claim 14 .  
     
     
         19 . An artificial neural network optimized according to  claim 15 .  
     
     
         20 . An artificial neural network optimized according to  claim 16 .  
     
     
         21 . An optimized artificial neural network, comprising: 
 a plurality of input neurons for receiving network input signals and for developing output signals in response thereto;    a plurality of output neurons receptive of signals for developing network output signals;    a plurality of hidden neurons which receive input signals and develop output signals;    signal transmission means comprised of a plurality of signal paths for applying output signals from said input neurons to said hidden neurons and for applying output signals from said hidden neurons to said output neurons; and    the number of hidden neurons, the neuron threshold functions and the signal path weights having values optimized by supervised learning and network modification by a genetic algorithm.    
     
     
         22 . A neural network according to  claim 21 , wherein one hidden neuron layer has only one neuron.