Deep learning approach for assessing credit risk

Abstract

Systems and methods to facilitate credit risk assessment are described herein. The systems and methods described herein relate to implementing and training a credit risk model comprising a document model and a company model. The document model may be configured to read text of a document, understand long range relationships between words, phrases, and the occurrence of one or more financial events, and create a document score that indicates whether the financial events are likely to occur based on that document. A document-model-state vector may be generated that represents important features and relationships identified within each document and across a set of documents for a given entity based on the document scores. The company model may produce a sequence of default probability scores representing overall likelihoods of the occurrence of the financial events for an entity based on the document-model-state vector for documents associated with that entity.

What is claimed is: 
     
         1 . A system configured to use a trained neural network to computationally assess the credit quality of an entity based at least in part on analyzing unstructured text from documents related to the entity to determine relationships between words and phrases that are indicative of one or more future financial events and utilizing a document model and a company model, the system comprising:
 one or more servers comprising electronic storage, one or more physical processors, and computer program components comprising a document model component including the document model configured to analyze unstructured text from documents, and a company model component configured to utilize the company model to produce a sequence of default probability scores representing overall likelihoods of the occurrence of one or more financial events for an entity, the one or more physical computer processors configured by computer readable instructions to:   electronically obtain a set of documents related to a first entity from one or more document sources over a network, the set of documents comprising at least a first document containing unstructured text and a second document containing unstructured text;   generate, utilizing the document model, document scores and document-representation vectors for at least the first and second documents of the set of documents based on analysis of the unstructured text, the document scores including a first document score for the first document of the set of documents and a second document score for the second document of the set of documents, the document-representation vectors including a first document-representation vector comprising a word embedding representation of the first document of the set of documents and a second document-representation vector comprising a word embedding representation of the second document of the set of documents, wherein the first and second document scores each indicate a likelihood of an occurrence of one or more future financial events for the first entity based on the first and second document, respectively;   aggregate the first and second document scores and the first and second document-representation vectors;   create, using the aggregated first and second document scores and first and second document-representation vectors, a document-model-state vector representing relationships identified within individual documents of the set of documents and across the set of documents;   produce, utilizing the company model, a sequence of default probability scores representing overall likelihoods of the occurrence of one or more future financial events based on the document-model-state vector for the set of documents related to the first entity.   
     
     
         2 . The system of  claim 1 , wherein the sequence of default probability scores are produced utilizing the document model and the company model. 
     
     
         3 . The system of  claim 1 , wherein the one or more processors are further configured to:
 generate a company score for the first entity based on the sequence of default probability scores, the company score comprising a value between zero (0) and one (1) and representing a default probability for the first entity.   
     
     
         4 . The system of  claim 1 , wherein the one or more processors are further configured to: aggregate the document-model-state vector with financial information for the entity, and wherein the one or more future financial events comprise one or more of default or bankruptcy. 
     
     
         5 . The system of  claim 1 , wherein the sequence of default probability scores is determined based on the aggregated document-model-state vector and financial information. 
     
     
         6 . The system of  claim 1 , wherein the one or more processors are further configured to:
 generate, utilizing the company model, an internal state vector based on the document-model-state vector, wherein the internal state vector comprises a first company-model-state vector.   
     
     
         7 . The system of  claim 6 , wherein the one or more processors are further configured to:
 input, into the company model, a second document-model-state vector and the first company-model-state vector, wherein the second document-model-state vector is based on aggregated document scores and document-representation vectors for a second set of documents;   produce, utilizing the company model, a second sequence of default probability scores representing overall likelihoods of the occurrence of one or more future financial events based on at least the second document-model-state vector; and   generate, utilizing the company model, a second internal state vector comprising a second company-model-state vector based on the second document-model-state vector and the first company-model-state vector.   
     
     
         8 . The system of  claim 1 , wherein the document model comprises:
 an input layer that receives at least the first and second documents of the set of documents;   an embedding layer creating at least the first and second document-representation vectors;   batch normalization that aggregates at least the first and second document-representation vectors; and   a GRU layer that generates the document-model-state vector, comprising a sequential network through which the aggregated document vectors are passed.   
     
     
         9 . The system of  claim 1 , wherein the document model comprises a deep-learning model with memory that reads the text from at least the first and second documents of the set of documents, understands long range relationships between words, phrases, and the occurrence of the one or more future financial events, and creates at least the first and second document scores. 
     
     
         10 . The system of  claim 1 , wherein the sequence of default probability scores further comprises a timeline of the overall likelihoods of the occurrence of one or more future financial events. 
     
     
         11 . A method for using a trained neural network to computationally assess the credit quality of an entity based at least in part on analyzing unstructured text from documents related to the entity to determine relationships between words and phrases that are indicative of one or more future financial events and utilizing a document model and a company model, the method comprising:
 storing computer program components comprising a document model component including the document model configured to analyze unstructured text from documents, and a company model component configured to utilize the company model to produce a sequence of default probability scores representing overall likelihoods of the occurrence of one or more financial events for an entity;   electronically obtaining a set of documents related to a first entity from one or more document sources over a network, the set of documents comprising at least a first document containing unstructured text and a second document containing unstructured text;   generating, utilizing the document model, document scores and document-representation vectors for at least the first and second documents of the set of documents based on analysis of the unstructured text, the document scores including a first document score for the first document of the set of documents and a second document score for the second document of the set of documents, the document-representation vectors including a first document-representation vector comprising a word embedding representation of the first document of the set of documents and a second document-representation vector comprising a word embedding representation of the second document of the set of documents, wherein the first and second document scores each indicate a likelihood of an occurrence of one or more future financial events for the first entity based on the first and second document, respectively;   aggregating the document scores and document-representation vectors;   creating, using the aggregated first and second document scores and first and second document-representation vectors, a document-model-state vector representing relationships identified within individual documents of the set of documents and across the set of documents;   producing, utilizing the company model, a sequence of default probability scores representing overall likelihoods of the occurrence of one or more future financial events based on the document-model-state vector for the set of documents related to the first entity.   
     
     
         12 . The method of  claim 11 , wherein the sequence of default probability scores are produced utilizing the document model and the company model. 
     
     
         13 . The method of  claim 11 , the method further comprising:
 generating a company score for the first entity based on the sequence of default probability scores, the company score comprising a value between zero (0) and one (1) and representing a default probability for the first entity.   
     
     
         14 . The method of  claim 11 , the method further comprising aggregating the document-model-state vector with financial information for the entity, and wherein the one or more future financial events comprise one or more of default or bankruptcy. 
     
     
         15 . The method of  claim 11 , wherein the sequence of default probability scores is determined based on the aggregated document-model-state vector and financial information. 
     
     
         16 . The method of  claim 11 , the method further comprising:
 generating, utilizing the company model, an internal state vector based on the document-model-state vector, wherein the internal state vector comprises a first company-model-state vector.   
     
     
         17 . The method of  claim 16 , the method further comprising:
 inputting a second document-model-state vector and the first company-model-state vector into the company model, wherein the second document-model-state vector is based on aggregated document scores and document-representation vectors for a second set of documents;   producing, utilizing the company model, a second sequence of default probability scores representing overall likelihoods of the occurrence of one or more future financial events based on at least the second document-model-state vector; and   generating, utilizing the company model, a second internal state vector comprising a second company-model-state vector based on the second document-model-state vector and the first company-model-state vector.   
     
     
         18 . The method of  claim 11 , wherein the document model comprises:
 an input layer that receives at least the first and second documents of the set of documents;   an embedding layer creating at least the first and second document-representation vectors;   batch normalization that aggregates at least the first and second document-representation vectors; and   a GRU layer that generates the document-model-state vector, comprising a sequential network through which the aggregated document vectors are passed.   
     
     
         19 . The method of  claim 11 , wherein the document model comprises a deep-learning model with memory that reads the text from at least the first and second documents of the set of documents, understands long range relationships between words, phrases, and the occurrence of the one or more future financial events, and creates at least the first and second document scores. 
     
     
         20 . The method of  claim 11 , wherein the sequence of default probability scores further comprises a timeline of the overall likelihoods of the occurrence of one or more future financial events.