Predicting accuracy of submitted data

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting the accuracy of user submissions. One of the methods includes receiving, from a user, an update to an attribute of an entity. If the user is determined to be reliable based on user profile data of the user, the knowledge base is updated with the update to the attribute of the entity.

1 . A system comprising:
 one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:   maintaining, by the system, a knowledge base accessible by multiple users, wherein the knowledge base comprises one or more attribute-value pairs for a plurality of entities;   obtaining, from a user having user profile data stored in the system, a value for an attribute of an entity of the plurality of entities, wherein the user profile data is not stored in the knowledge base;   obtaining the user profile data for the user, the user profile data including information representing other subsystems of the system accessed by the user;   computing a likelihood that the value is accurate using the information representing the other subsystems; and   updating the knowledge base with the value in response to determining that the likelihood satisfies a threshold.   
     
     
         2 . The system of  claim 1 , wherein the likelihood that the value is accurate is output from a user reliability model, in response to an input of the information representing the other subsystems. 
     
     
         3 . The system of  claim 2 , wherein the user reliability model is configured to consider users who access more subsystems of the system to be more reliable than users who access fewer subsystems of the system. 
     
     
         4 . The system of  claim 2 , wherein the user reliability model is trained using training examples comprising respective subsystems of the system accessed by each user of a plurality of users and a measure of accuracy of previously submitted updates to the knowledge base by the each user. 
     
     
         5 . The system of  claim 1 , wherein the other subsystems include an image search system, a map system, an email system, a social network system, a blogging system, or a shopping system. 
     
     
         6 . The system of  claim 1 , wherein the other subsystems of the system do not include a search engine or the knowledge base. 
     
     
         7 . The system of  claim 1 , wherein the updating of the knowledge base with the value comprises automatically updating the knowledge base without intervention or inspection by a knowledge base administrator. 
     
     
         8 . A method performed by a search system comprising one or more computers, the method comprising:
 maintaining, by the search system, a knowledge base accessible by multiple users, wherein the knowledge base comprises information about entities, the information about each entity of the entities being represented as one or more attribute-value pairs;   receiving, by the search system from a user having user profile data stored in the search system, an updated value of an attribute of an entity in the knowledge base, wherein the user profile data is not stored in the knowledge base;   obtaining the user profile data for the user, the user profile data including information representing other subsystems of the search system accessed by the user;   computing a likelihood that the updated value is accurate using the information representing the other subsystems of the search system; and   updating the knowledge base with the updated value in response to determining that the likelihood satisfies a threshold.   
     
     
         9 . The method of  claim 8 , wherein the likelihood that the updated value is accurate is output from a user reliability model, in response to an input of the information representing the other subsystems. 
     
     
         10 . The method of  claim 9 , wherein the user reliability model is configured to consider users who access more subsystems of the search system to be more reliable than users who access fewer subsystems of the search system. 
     
     
         11 . The method of  claim 9 , wherein the user reliability model is trained using training examples comprising respective subsystems of the search system accessed by each user of a plurality of users and a measure of accuracy of previously submitted updates to the knowledge base by the each user. 
     
     
         12 . The method of  claim 8 , wherein the other subsystems include an image search system, a map system, an email system, a social network system, a blogging system, or a shopping system. 
     
     
         13 . The method of  claim 8 , wherein the other subsystems of the search system do not include a search engine or the knowledge base. 
     
     
         14 . The method of  claim 8 , wherein the updating of the knowledge base with the updated value comprises automatically updating the knowledge base without intervention or inspection by a knowledge base administrator. 
     
     
         15 . The method of  claim 8 , wherein the receiving of the updated value in the knowledge base comprises receiving the updated value by a web search engine of the search system. 
     
     
         16 . The method of  claim 8 , wherein the receiving of the updated value comprises receiving the updated value through a knowledge panel user interface provided by the web search engine in response to the user submitting a search query. 
     
     
         17 . The method of  claim 8 , wherein the knowledge panel user interface presents one or more items of information about the entity in the knowledge base. 
     
     
         18 . One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers of a search system cause the one or more computers to perform operations comprising:
 maintaining, by the search system, a knowledge base accessible by multiple users, wherein the knowledge base comprises information about entities, the information about each entity of the entities being represented as one or more attribute-value pairs;   receiving, by the search system from a user having user profile data stored in the search system, a value of an attribute of an entity in the knowledge base, wherein the user profile data is not stored in the knowledge base;   obtaining the user profile data for the user, the user profile data including information representing other subsystems of the search system accessed by the user, wherein the other subsystems include an image search system, a map system, an email system, a social network system, a blogging system, or a shopping system;   computing a likelihood that the value is accurate using the information representing the other subsystems; and   updating the knowledge base with the updated value received from the user in response to determining that the likelihood satisfies a threshold.   
     
     
         19 . The one or more non-transitory computer storage media of  claim 18 , wherein the likelihood that the value is accurate is output from a trained user reliability model, in response to an input of the information representing the other subsystems. 
     
     
         20 . The one or more non-transitory computer storage media of  claim 19 , wherein the trained user reliability model is configured to consider users who access more subsystems of the search system to be more reliable than users who access fewer subsystems of the search system.