System and method for determining substitutes for a requested product and the order to provide the substitutes

Abstract

A system can include one or more processors and non-transitory computer-readable media storing computing instructions configured to run on the one or more processors to cause the one or more processors to perform: extracting one or more visual attributes of a requested product in a domain based on an image describing the requested product in an inquiry from a user; extracting one or more textual attributes of the requested product based on a text describing the requested product in the inquiry; determining projected requested products, in a product graph relating to the domain and dynamically generated by machine learning, based on the one or more visual attributes of the requested product and the one or more textual attributes of the requested product; determining one or more substitutes for the requested product based on scores of candidate products in the product graph, wherein: a greater weight is given to visual distances than textual distances in determining the scores; the visual distances and the textual distances comprise distances between the projected requested products and the candidate products in the product graph; and the distances are based on vectors between the projected requested products and the candidate products as projected in each dimension of multiple dimensions of a multi-dimensional space of the product graph; and automatically transmitting, via a computer network and to a user device for the user, information about higher ranked ones of the one or more substitutes to facilitate a display of the information on a graphical user interface on the user device. Other embodiments are disclosed.

What is claimed is: 
     
         1 . A system comprising:
 one or more processors; and   one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors to cause the one or more processors to perform:
 extracting one or more visual attributes of a requested product in a domain based on an image describing the requested product in an inquiry from a user; 
 extracting one or more textual attributes of the requested product based on a text describing the requested product in the inquiry; 
 determining projected requested products, in a product graph relating to the domain and dynamically generated by machine learning, based on the one or more visual attributes of the requested product and the one or more textual attributes of the requested product; 
 determining one or more substitutes for the requested product based on scores of candidate products in the product graph, wherein:
 a greater weight is given to visual distances than textual distances in determining the scores; 
 the visual distances and the textual distances comprise distances between the projected requested products and the candidate products in the product graph; and 
 the distances are based on vectors between the projected requested products and the candidate products as projected in each dimension of multiple dimensions of a multi-dimensional space of the product graph; and 
 automatically transmitting, via a computer network and to a user device for the user, information about higher ranked ones of the one or more substitutes to facilitate a display of the information on a graphical user interface on the user device. 
 
   
     
     
         2 . The system in  claim 1 , wherein the computing instructions are further configured to run on the one or more processors to cause the one or more processors to perform:
 extracting one or more color attributes of the requested product based on the image and the text; and   determining a color distance of a candidate substitute of the one or more substitutes based on the one or more color attributes of the requested product and one or more color entities of the candidate substitute of the one or more substitutes.   
     
     
         3 . The system in  claim 2 , wherein:
 the one or more color attributes of the requested product comprises a product major color and a product minor color; and   the one or more color entities of the candidate substitute of the one or more substitutes comprise a candidate major color and a candidate minor color.   
     
     
         4 . The system in  claim 2 , wherein the computing instructions are further configured to run on the one or more processors to cause the one or more processors to perform:
 determining a context of the inquiry, wherein:
 the context of the inquiry includes at least one of an address of the user, a purchase history of the user, another term in the inquiry of the user, a style identified in the image, or another object identified in the image; 
 the extracting the one or more visual attributes of the requested product further comprises extracting the one or more visual attributes based on the context; 
 the extracting the one or more textual attributes of the requested product further comprises extracting the one or more textual attributes based on the context; and 
 the extracting the one or more color attributes of the requested product further comprises extracting the one or more color attributes based on the context. 
   
     
     
         5 . The system in  claim 1 , wherein:
 the product graph comprises:
 one or more product nodes, wherein:
 the one or more product nodes comprise the one or more substitutes; 
 
 one or more entity nodes, wherein:
 the one or more entity nodes comprise the one or more visual attributes and the one or more textual attributes of the requested product; 
 
 one or more entity-entity connections between each pair of related entity nodes of the one or more entity nodes; 
 one or more product-entity connections, wherein:
 each of the one or more product-entity connections is between a respective product node of the one or more product nodes and a respective entity node of the one or more entity nodes; 
 the respective entity node is related to the respective product node within a same one of the one or more product-entity connections; and 
 the each of the one or more product-entity connections comprises a respective product-entity connection distance determined based on a respective relatedness degree between the respective product node and the respective entity node; and 
 one or more product-product connections between a respective first product node of the one or more product nodes and a respective second product node of the one or more product nodes. 
 
   
     
     
         6 . The system in  claim 5 , wherein:
 each of the one or more product nodes comprises a respective product coordinate in the multi-dimensional space; and   each of the one or more entity nodes comprises a respective entity coordinate in the multi-dimensional space.   
     
     
         7 . The system in  claim 5 , wherein the computing instructions are further configured to run on the one or more processors to cause the one or more processors to perform one or more of:
 dynamically learning, by machine learning, information about the domain from one or more expert opinions, an inventory database, and public records;   creating, by machine learning, one or more product categories in the domain based on the information about the domain;   extracting, by machine learning, one or more first entity nodes from the one or more product categories based on the information about the domain;   creating, by machine learning, one or more second entity nodes based on the one or more first entity nodes, each of the one or more second entity nodes being related to at least one of the one or more first entity nodes according to a first set of criteria, wherein:
 the one or more entity nodes comprise the one or more first entity nodes and the one or more second entity nodes; 
   building, by machine learning, an entity-entity connection, of the one or more entity-entity connections, between two related entity nodes of the one or more entity nodes, when the two related entity nodes are either associated with one of the one or more product categories or related to each other according to the first set of criteria;   building, by machine learning, a product-entity connection (with the respective product-entity connection distance) of the one or more product-entity connections, between a first entity node of the one or more entity nodes and a first product node of the one or more product nodes, based on the information about the domain, wherein:
 the first product node is related to the first entity node; and 
 the respective product-entity connection distance of the product-entity connection is determined based on the respective relatedness degree between the first entity node and the first product node; or 
   building, by machine learning, a product-product connection of the one or more product-product connections, between the first product node and a second product node selected from the one or more product nodes when the first product node and the second product node are determined to be substitutable by or complimentary of each other based on the information about the domain.   
     
     
         8 . The system in  claim 7 , wherein the computing instructions are further configured to run on the one or more processors to cause the one or more processors to perform:
 providing a user interface configured to be executed on one or more user computers for one or more advanced users to receive the one or more expert opinions from the one or more advanced users, wherein:
 the public records include at least one of product catalogs, publications, product reviews, blog entries, online discussion threads, search keywords, curated collections, or purchase patterns. 
   
     
     
         9 . The system in  claim 7 , wherein one or more of:
 the first set of criteria used by machine learning to determine that the first entity node of the one or more entity nodes is related to the second entity node of the one or more entity nodes comprises one of:
 the first entity node is linguistically similar to the second entity node; 
 the first entity node is linguistically opposite to the second entity node; or 
 the first entity node is linguistically inferable from the second entity node; or 
   a distance value of each of the entity-entity connection and the product-product connection is assigned by machine learning.   
     
     
         10 . The system in  claim 5 , wherein:
 determining the projected requested products further comprises:
 determining a first projected requested product, in the product graph based on the one or more visual attributes of the requested product; and 
 determining a second projected requested product in the product graph based on the one or more textual attributes of the requested product, the projected requested products comprising the first projected requested product and the second projected requested product; and 
   determining the one or more substitutes for the requested product further comprises:
 determining first candidates of the one or more product nodes, each of the first candidates comprising one of a first predetermined number of lowest visual distances from the first projected requested product among one or more visual distances of the one or more product nodes from the first projected requested product; and 
 determining second candidates of the one or more product nodes, each of the second candidates comprising one of a second predetermined number of lowest textual distances from the second projected requested product among one or more textual distances of the one or more product nodes from the second projected requested product, the candidate products comprising the first candidates and the second candidates. 
   
     
     
         11 . A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:
 extracting one or more visual attributes of a requested product in a domain based on an image describing the requested product in an inquiry from a user;   extracting one or more textual attributes of the requested product based on a text describing the requested product in the inquiry;   determining projected requested products, in a product graph relating to the domain and dynamically generated by machine learning, based on the one or more visual attributes of the requested product and the one or more textual attributes of the requested product;   determining one or more substitutes for the requested product based on scores of candidate products in the product graph, wherein:
 a greater weight is given to visual distances than textual distances in determining the scores; 
 the visual distances and the textual distances comprise distances between the projected requested products and the candidate products in the product graph; and 
 the distances are based on vectors between the projected requested products and the candidate products as projected in each dimension of multiple dimensions of a multi-dimensional space of the product graph; and 
   automatically transmitting, via a computer network and to a user device for the user, information about higher ranked ones of the one or more substitutes to facilitate a display of the information on a graphical user interface on the user device.   
     
     
         12 . The method in  claim 11 , further comprising:
 extracting one or more color attributes of the requested product based on the image and the text; and   determining a color distance of a candidate substitute of the one or more substitutes based on the one or more color attributes of the requested product and one or more color entities of the candidate substitute of the one or more substitutes.   
     
     
         13 . The method in  claim 12 , wherein:
 the one or more color attributes of the requested product comprises a product major color and a product minor color; and   the one or more color entities of the candidate substitute of the one or more substitutes comprise a candidate major color and a candidate minor color.   
     
     
         14 . The method in  claim 12 , further comprising:
 determining a context of the inquiry, wherein:
 the context of the inquiry includes at least one of an address of the user, a purchase history of the user, another term in the inquiry of the user, a style identified in the image, or another object identified in the image; 
 the extracting the one or more visual attributes of the requested product further comprises extracting the one or more visual attributes based on the context; 
 the extracting the one or more textual attributes of the requested product further comprises extracting the one or more textual attributes based on the context; and 
 the extracting the one or more color attributes of the requested product further comprises extracting the one or more color attributes based on the context. 
   
     
     
         15 . The method in  claim 11 , wherein:
 the product graph comprises:
 one or more product nodes, wherein:
 the one or more product nodes comprise the one or more substitutes; 
 
 one or more entity nodes, wherein:
 the one or more entity nodes comprise the one or more visual attributes and the one or more textual attributes of the requested product; 
 
 one or more entity-entity connections between each pair of related entity nodes of the one or more entity nodes; 
 one or more product-entity connections, wherein:
 each of the one or more product-entity connections is between a respective product node of the one or more product nodes and a respective entity node of the one or more entity nodes; 
 the respective entity node is related to the respective product node within a same one of the one or more product-entity connections; and 
 the each of the one or more product-entity connections comprises a respective product-entity connection distance determined based on a respective relatedness degree between the respective product node and the respective entity node; and 
 
 one or more product-product connections between a respective first product node of the one or more product nodes and a respective second product node of the one or more product nodes. 
   
     
     
         16 . The method in  claim 15 , wherein:
 each of the one or more product nodes comprises a respective product coordinate in the multi-dimensional space; and   each of the one or more entity nodes comprises a respective entity coordinate in the multi-dimensional space.   
     
     
         17 . The method in  claim 15 , further comprising one or more of:
 dynamically learning, by machine learning, information about the domain from one or more expert opinions, an inventory database, and public records;   creating, by machine learning, one or more product categories in the domain based on the information about the domain;   extracting, by machine learning, one or more first entity nodes from the one or more product categories based on the information about the domain;   creating, by machine learning, one or more second entity nodes based on the one or more first entity nodes, each of the one or more second entity nodes being related to at least one of the one or more first entity nodes according to a first set of criteria, wherein:
 the one or more entity nodes comprise the one or more first entity nodes and the one or more second entity nodes; 
   building, by machine learning, an entity-entity connection, of the one or more entity-entity connections, between two related entity nodes of the one or more entity nodes, when the two related entity nodes are either associated with one of the one or more product categories or related to each other according to the first set of criteria;   building, by machine learning, a product-entity connection (with the respective product-entity connection distance) of the one or more product-entity connections, between a first entity node of the one or more entity nodes and a first product node of the one or more product nodes, based on the information about the domain, wherein:
 the first product node is related to the first entity node; and 
 the respective product-entity connection distance of the product-entity connection is determined based on the respective relatedness degree between the first entity node and the first product node; or 
   building, by machine learning, a product-product connection of the one or more product-product connections, between the first product node and a second product node selected from the one or more product nodes when the first product node and the second product node are determined to be substitutable by or complimentary of each other based on the information about the domain.   
     
     
         18 . The method in  claim 17 , further comprising:
 providing a user interface configured to be executed on one or more user computers for one or more advanced users to receive the one or more expert opinions from the one or more advanced users, wherein:
 the public records include at least one of product catalogs, publications, product reviews, blog entries, online discussion threads, search keywords, curated collections, or purchase patterns. 
   
     
     
         19 . The method in  claim 17 , wherein one or more of:
 the first set of criteria used by machine learning to determine that the first entity node of the one or more entity nodes is related to the second entity node of the one or more entity nodes comprises one of:
 the first entity node is linguistically similar to the second entity node; 
 the first entity node is linguistically opposite to the second entity node; or 
 the first entity node is linguistically inferable from the second entity node; or 
   a distance value of each of the entity-entity connection and the product-product connection is assigned by machine learning.   
     
     
         20 . The method in  claim 15 , wherein:
 determining the projected requested products further comprises:
 determining a first projected requested product, in the product graph based on the one or more visual attributes of the requested product; and 
 determining a second projected requested product in the product graph based on the one or more textual attributes of the requested product, the projected requested products comprising the first projected requested product and the second projected requested product; and 
   determining the one or more substitutes for the requested product further comprises:
 determining first candidates of the one or more product nodes, each of the first candidates comprising one of a first predetermined number of lowest visual distances from the first projected requested product among one or more visual distances of the one or more product nodes from the first projected requested product; and 
 determining second candidates of the one or more product nodes, each of the second candidates comprising one of a second predetermined number of lowest textual distances from the second projected requested product among one or more textual distances of the one or more product nodes from the second projected requested product, the candidate products comprising the first candidates and the second candidates.