Neural-symbolic computing

Abstract

A neural-symbolic computing engine can have two or more modules that are configured to cooperate with each other in order to create one or more gradient-based machine learning models that use machine learning on i) knowledge representations and ii) reasoning to solve an issue. A model representation module in the neural-symbolic computing engine is configured to apply one or more mathematical functions, at least including a logit transform, to truth values from first order logic elements supplied from a language module of the neural-symbolic computing engine.

What is claimed is: 
     
       1. An apparatus, comprising:
 a neural-symbolic computing engine that has two or more modules that are configured to cooperate with each other in order to create one or more gradient-based machine learning models that use machine learning on i) knowledge representations and ii) reasoning, to solve an issue, where a model representation module in the neural-symbolic computing engine is configured to apply one or more mathematical functions, at least including a logit transform, to truth values from first order logic elements supplied from a language module of the neural-symbolic computing engine. 
 
     
     
       2. The apparatus of  claim 1 , where the two or more modules further cooperate to cause an encoding of the knowledge representations and the reasoning into a first gradient-based machine learning model from information supplied by a person to a theory module of the neural-symbolic computing engine. 
     
     
       3. The apparatus of  claim 1 , where a theory representation module in the neural-symbolic computing engine is configured to compile a neural network as a first gradient-based machine learning model that undergoes gradient-based learning. 
     
     
       4. The apparatus of  claim 3 , where the first gradient-based machine learning model that undergoes gradient-based learning is configured to i) adapt vectors associated with different constants of a set of rules that act as constraints and ii) adapt vectors of parameters of a network associated with different functions and predicates, in order to obtain an interpretation that makes the rules as true as possible. 
     
     
       5. The apparatus of  claim 1 , where the model representation module is further configured to create a first gradient-based machine learning model that uses the logit transform of the truth values to avoid vanishing gradients that would have resulted from multiplying truth values of a network in the first gradient-based machine learning model. 
     
     
       6. The apparatus of  claim 5 , where the truth values come from a network structure in the network having multiple intermediate levels from a bottom of the network structure to a top level of the network structure. 
     
     
       7. The apparatus of  claim 1 , where the one or more gradient-based machine learning models that use machine learning are one or more neural networks, where the neural-symbolic computing engine is configured to represent the knowledge representations and reasoning in the first order logic, where relations and functions that make up a vocabulary of the knowledge representations and reasoning are then implemented in the one or more neural networks that can have an arbitrary network structure, where logical connectives in the knowledge representations and reasoning are composed into a single deep network with multiple intermediate levels from a bottom of the network structure to a top level of the network structure, which is trained to maximize a truthfulness measure of the knowledge representations and reasoning. 
     
     
       8. The apparatus of  claim 1 , where a learner algorithm module in the neural-symbolic computing engine has a side-rules component configured to provide an architecture to act as a flexible gate within a framework of a first gradient-based machine learning model using the first order logic in which the knowledge representations are used as fixed rules, and in some situations, the side-rules component modifies application of these fixed rules to influence learning in the first gradient-based machine learning model. 
     
     
       9. The apparatus of  claim 1 , where the model representation module in the neural-symbolic computing engine is further configured to apply the logit transform to truth values from the first order logic elements so that a first gradient-based machine learning model is able to consider any amount of conjunctions of features while learning and solving the issue as a whole; rather than having to break down parts of that issue being solved into smaller sub problems. 
     
     
       10. A non-transitory computer-readable medium including executable instructions that, when executed with one or more processors, cause a neural-symbolic computing system to perform operations as follows, comprising:
 creating one or more gradient-based machine learning models that use machine learning on i) knowledge representations and ii) reasoning to solve an issue; and 
 applying one or more mathematical functions, at least including a logit transform, to truth values from first order logic elements. 
 
     
     
       11. A method for neural-symbolic computing, comprising:
 configuring a neural-symbolic computing engine to create one or more gradient-based machine learning models that use machine learning on i) knowledge representations and ii) reasoning to solve an issue, where the neural-symbolic computing engine applies one or more mathematical functions, at least including a logit transform, to truth values from first order logic elements. 
 
     
     
       12. The method of  claim 11 , further comprising:
 causing an encoding of the knowledge representations and the reasoning into a first gradient-based machine learning model from information supplied by a person to the neural-symbolic computing engine. 
 
     
     
       13. The method of  claim 11 , where the neural-symbolic computing engine is configured to compile a neural network as a first gradient-based machine learning model that undergoes gradient-based learning. 
     
     
       14. The method of  claim 13 , further comprising:
 configuring the first gradient-based machine learning model that undergoes gradient-based learning to i) adapt vectors associated with different constants of a set of rules that act as constraints and ii) adapt vectors of parameters of a network associated with different functions and predicates, in order to obtain an interpretation that makes the rules as true as possible. 
 
     
     
       15. The method of  claim 11 , further comprising:
 creating a first gradient-based machine learning model that uses the logit transform of the truth values to avoid vanishing gradients that would have resulted from multiplying truth values of a network in the first gradient-based machine learning model. 
 
     
     
       16. The method of  claim 15 , where the truth values come from a network structure in the network having multiple intermediate levels from a bottom of the network structure to a top level of the network structure. 
     
     
       17. The method of  claim 11 , where the one or more gradient-based machine learning models that use machine learning are one or more neural networks, where the neural-symbolic computing engine is configured to represent the knowledge representations and reasoning in the first order logic, where relations and functions that make up a vocabulary of the knowledge representations and reasoning are then implemented in the one or more neural networks that can have an arbitrary network structure, where logical connectives in the knowledge representations and reasoning are composed into a single deep network with multiple intermediate levels from a bottom of the network structure to a top level of the network structure, which is trained to maximize a truthfulness measure of the knowledge representations and reasoning. 
     
     
       18. The method of  claim 11 , further comprising:
 configuring side-rules to provide an architecture to act as a flexible gate within a framework of a first gradient-based machine learning model using the first order logic in which the knowledge representations are used as fixed rules, and in some situations, the side-rules modify application of these fixed rules to influence learning in the first gradient-based machine learning model. 
 
     
     
       19. The method of  claim 11 , further comprising:
 configuring the neural-symbolic computing engine to use the first order logic to create fixed rules to guide learning in a first gradient-based machine learning model, and 
 configuring side-rules to mask specified information in specified situations with respect to the fixed rules to influence learning in the first gradient-based machine learning model. 
 
     
     
       20. The method of  claim 11 , further comprising:
 configuring the neural-symbolic computing engine to apply the logit transform to truth values from the first order logic elements so that a first gradient-based machine learning model is able to consider any amount of conjunctions of features while learning and solving the issue as a whole; rather than having to break down parts of that issue being solved into smaller sub problems each with its own sub set of conjunctions of features and having to learn and train solve each one of those sub problems individually and then subsequently having to train to use the sub problems' combined outputs to solve the issue as the whole.